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Introduction 


On May 10-12, 1995, the formal methods team at the NASA Langley Research Center 
sponsored their third workshop 1 on the application of formal methods to the design and 
verification of life-critical systems. This workshop brought together formal methods 
researchers, industry engineers, and academicians to discuss the potential of NASA-spon- 
sored formal methods and to investigate new opportunities for applying these methods to 
industry problems. 

This publication constitutes the proceedings of the workshop. It contains copies of the 
material presented at the workshop, summaries of many of the presentations, a complete 
list of attendees, and a detailed summary of the Langley formal methods program. Much of 
the material contained herein is available electronically through the World-Wide Web via 
the following URL: http://atb-www.larc.nasa.gov/WS95/proceedings.html. 

On behalf of the formal methods team, we thank all of the presenters and attendees for 
their contributions to making this workshop a great success. We look forward to seeing all 
of you again at our next workshop. 


C. Michael Holloway, Workshop Co-chairman 
Ricky W. Butler, Workshop Co-chairman 


1 Previous workshops were held in 1990 (see NASA Conference Publication 10052) and 1992 (see NASA Conference Publication 
10110 ). 


PRECEDING PAGE BLANK NOT FLMED 

PAGE — 2__ INTENTIONALLY BLANK 


Final Agenda 


PRECEDING PAGE BLANK NOT HLMED 


PAGE. 


4—i 


NTENTiC 





Third NASA Langley 
Formal Methods Workshop 



NASA Langley 


May 10-12 , 1995 

H. J. E. Reid Conference Center 
NASA Langley Research Center 
Hampton , Virginia 


Sponsored by 
Formal Methods Team 
Assessment Technology Branch 
Information & Electromagnetic Technology Division 
Research and Technology Group 


Proceedings available on the World- Wide Web at 
http ://atb-www.larc.nasa.gov/W S95/proceedings.html 


PRECEDING PAGE BLANK NOT FILStfOJ 

; tv % 


Wednesday, May 10, 1995 


8:00 

8:45 

9:00 

9:30 

10:30 

11:00 

12:00 

12:30 

1:30 

2:00 

2:30 

3:00 

3:30 

5:00 


8:45 Late Registration 

Session 1: Introduction to Formal Methods 
Chaired by Ricky Butler 

9:00 Welcome 

Milton Holt, Chief, Information & Electromagnetic Technology Division 

9:30 Rationale for Formal Methods 

Ricky Butler, NASA Langley Research Center 

10:30 An Informal Introduction to Formal Methods 

Michael Holloway, Ricky Butler, and Paul Miner 
NASA Langley Research Center 

11:00 Break 

12:00 An Informal Introduction to Formal Methods (continued) 

12:30 Overview of NASA Langley’s Formal Methods Program 

Ricky Butler, NASA Langley Research Center 

1 :30 Lunch in NASA Cafeteria 

Session 2: LaRC -sponsored Industrial Applications 
Chaired by Ricky Butler 

2:00 The AAMP5/AAMP-FV Project 

Steve Miller, Rockwell-Collins 
Mandayam Srivas, SRI International 

2:30 Union Switch & Signal Project 

Joe Profeta, Union Switch & Signal 
Doug Hoover, Odyssey Research Associates 

3:00 Honeywell Software Development Project 

Lance Sherry, Honeywell 

Doug Hoover, Odyssey Research Associates 

3:30 Break 

Session 3: Industry Perspectives on Formal Methods 
Chaired by Michael Holloway 

5:00 Panel Session 

Scott French, Loral ATC 
Steve Miller, Rockwell-Collins 
Joe Profeta, Union Switch & Signal 
Lance Sherry, Honeywell 

6:00 Cash Bar Social in Reid Conference Center 


Thursday, May 11, 1995 


8:30 - 9:30 
9:30 - 10:30 

10:30 - 10:45 

10:45 - 11:45 
11:45 - 12:30 

12:30 - 1:30 

1:30 - 2:00 
2:00 - 2:30 
2:30 - 3:15 
3:15 - 3:30 

3:30 - 5:00 
6:30 - 8:00 


Session 4: Software Systems (1) 

Chaired by Ricky Butler 

Formal Verification for Fault-Tolerant Architectures/PVS Design 

John Rushby, SRI International 

Formal Methods Demonstration Project for Space Applications 

Ben DiVito, Vl'GYAN, Inc. 

John Kelly, Jet Propulsion Laboratory 

Break 

Session 5: Software Systems (2) 

Chaired by Michael Holloway 

Ada 9X Language Precision Team 

David Guaspari, Odyssey Research Associates 

Introduction to Penelope and Its Applications 

David Guaspari, Odyssey Research Associates 

Lunch in NASA Cafeteria 

Session 6: Hardware Systems 
Chaired by Paul Miner 

The Formal Verification Technology Used on AAMP5 

Mandayam Srivas, SRI International 

Specification and Verification of VHDL Designs 

Damir Jamsek, Odyssey Research Associates 

Derivational Reasoning System 

Bhaskar Bose, Derivation Systems Inc. 

Break 

Session 7: Researcher Perspectives on Formal Methods 
Chaired by Jim Caldwell 

Panel Session 

David Dill, Stanford University 
Doug Hoover, Odyssey Research Associates 
Steve Johnson, Indiana University 
Natarajan Shankar, SRI International 

Dinner at Captain George’s Seafood Restaurant 



Friday, May 12, 1995 


8:30 

9:15 

10:00 

10:30 

11:15 

12:00 

1:30 

2:00 

2:30 


Session 8: Research Issues (1) 

Chaired by Michael Holloway 

9:15 Safety Analysis 

John Knight, University of Virginia 

10:00 Non-standard Analysis and Software 

Richard Platek, Odyssey Research Associates 

10:30 Break 


Session 9: Research Issues (2) 
Chaired by Victor Carrefio 

11:15 Hybrid Fault Algorithms 

Pat Lincoln, SRI International 

12:00 Model Checking 

David Dill, Stanford University 


1:30 Lunch in NASA Cafeteria 

Session 10: Research Issues (3) 

Chaired by Ricky Butler 

2:00 The DDD Scheme Machine 

Steve Johnson, Indiana University 

2:30 Formal Development of a Clock Synchronization Circuit 

Paul Miner, NASA Langley Research Center 

2:40 Closing Remarks 

Ricky Butler, NASA Langley Research Center 


10 


Session 1: Introduction to Formal Methods 

Ricky W. Butler, Chair 


® Welcome, by H. Milton Holt, Chief, Information & Electromagnetic Technology Division 

• Rationale for Formal Methods, by Ricky W. Butler 

• An Informal Introduction to Formal Methods, by C. Michael Holloway, Paul S. Miner, and Ricky 
W. Butler 

® Overview of NASA Langley’s Formal Methods Program, by Ricky W. Butler 
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finite testing finite testing inadequate, 

• Have to separately reason about or test millions of sequences of 

discrete state transitions and interpolation OK interpolation unsound 



Testing (Lots of it) 

Design Diversity (i.e. Software Fault-Tolerance: 
version programming, recovery blocks, etc.) 


z 
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( 10 9 hours — 114,000 years) 






FM Versus Traditional Engineering of Digital Systems 
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Formal Methods: Specification Formal Methods: Verification 
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1. Formal methods offers the most intellectually 
defensible means of producing life-critical systems 
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-- Perhaps motivated future in-depth study of formal methods 

. The remaining presentations should continue to motivate 
and to set context, and should also 
-- Demonstrate the variety of formal techniques that are available 
-- Illustrate ways in which formal methods may be applied in various 
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for overcoming them. 
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(1 - p )(*2 “ M < £>(*2) - < (1 + p )(*2 _ * l ) 

2 . There is a 6 such that if clocks C v and C q are non-faulty at time t, then: 

\C P (t)-C q (t)\<& 
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the corresponding operator in real arithmetic. 
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real_to_fp( (fp_to_real(/inl) + fp_to_real(/in2)) , mode ) 
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Top-level: Abstract description of system (and assumptions) 

Lower-level: Detailed description of system (and assumptions) 

Prove: The detailed system description has the same behavior as the abstract descrip- 
tion given the assumptions and an abstraction function relating the two systems. 




Asynchronous processors 
Clock time and real lime 
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Obstacles to getting started support 
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Specifying the Flight Schedule Database Accessing an Entry 
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addjlight(djlt>8ched)(x) 



scheduled?(add^ftight(d , fit, sched )) = 

delete -flight( IF scheduled!^ fit) THEN d schcdtiledl{ IF scheduled! {d, fit) THEN d 

ELSE d WITH (/// := sched] ENDIF ) ELSE d WITH [fit := sched] ENDIF 
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This type of trivial problem is usually not manifested until when one attempts a of operations a property holds, 

mechanical (i.e. level 3) verification. 

• Testing can never establish this. 
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TYPE - [# X RECORD 

departure_tm: Dat«_and_time , 
arrival_tm: Date.and.tine . 
dep.gate : Gate.nums , 
arrogate: Gate_nums, 
arr_or_dep: A.or.D, 
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• Initiate demonstration projects in problem domains where current 
formal methods are adequate. 



Scope of Program is Large Technology TVansfer Strategy 
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Some Lessons Learned (Continued) NASA Langley’s Formal Methods (FM) Program 


A 

u 

? 

« 

a 

.2 

i 

v 

V 


S ® £ 

£ .i S 

« a 
co S £ 

o S &| 

r n ttO © 

0)0 CC S 


s 

fc» 

«s 

© 

I ^ 

5 CQ 

1 . cT 

S 

u .5 


a ® 
a > g 

5 £ 

a 

•o <s 

ss 


? a » 

Jn on 


s 2 


£ 5 ® 


.2 a 

>» o 
« 
a 


a a 
© 

Js a 

<J V 

© w 


X 

< s 

< *0 
2 a 
~T 3 


CL _ 


H ^ 


* ^ c 

*13 U M U 


g « f s 


03 a 


II 

*2 - 
'u u 
-o •£. 

« £ 


x s 


S 2 

u 

£ « 
m 

0) © 

A * 

H 5 


!! 
i « 

a> >> 

5 *3 

H o 


S 

.2 b 
bo v 
a > 

— o 

s s 

- 0 


s E 

'~T « 

0 CL 

3 « 

1 e 

0 - LX 

3 * 2-2 
E « “ H 


II 


E > ^ 0 , * z » *2 


It! 

2l< 

4) » •> 


I 5 

O JZ — •* 

« u O a) 


= * 


» I 

£> * 
1 -8 


o o o e o o 


a 

D 


• s S 

S J 
»“ « 
*0 _ 
8| 

I- 

> O 


11 

§1 

•* x 

b® © 
o _ 

S ..2 


g 

CO g 

< I 


2 c 

A cd 


O — 
v "0 

18 

0 .. 

bO co 
-O ^ 

0) o 

1 £ 
c 1 

X c 

» T3 *3 

d .2 

0 B *» 

X u c 

£ o 8 
*8 — ,2 
2* fl £ 


0 s _jj 
U 0 *r> 

j n 

+* a v 

« J5 

5 « 5 

1 -2 .£ 

flee 
o •- Cd 

CO 5 Q, 

uc.tr 

<y C y 

« ^ s 

to £ a 

£ £ v 

'"V 3 > 

s o ^ 
•- u o 
Pu a cc 


# u o 
X 
X ^ 
-** a> 

^ t? 

ffl" 2 

4 x 

A P 


2 - S.g 

3 * ® 


? s 


= S 

s s 

2 = 
o a 

■S S 

s-§ 

G w 
tn e 
0) B 
u <y 


m ^ .2 

T 3 O W 
0 u ^ 

-c a *3 

15 « *2 


•S o 

£ « 
U "fj 

^ J 
0 *> 
J3 « 

H E 


T3 

O 

0) 

a 


cd 

* 

T3 

L 

cd 

X 

.£ *3 

« .2 
V ^ 
CO o 

cs a 

0 ) Ch 

u n 
u r 

.S .5 

o 

s ^ 

U 53 

ce ^ 
C tn 
M-o 
to "C 

i | 

>L* ^ 

-w co 
3 *3 
O O 


£ £ 


J2 

cd 

0 


01 

X 


e 

.2 

t*3 

u 

3 

TJ 

V 

•° L 

u o 
cd o 

E a 

1 s 

3 o 

Cd b 

^ fl 

C « 
.2 £ 
o ^ 

£ <2 
a o 


56 


Session 2: LaRC-sponsored Industrial 
Applications 

Ricky W. Butler, Chair 


® The AAMP5/AAMP-FV Project by Steve Miller, Rockwell-Collins 

• Union Switch & Signal Project, by Joe Profeta, Union Switch & Signal; and Doug Hoover, 
Odyssey Research Associates 

® Honeywell Software Development Project, by Lance Sherry, Honeywell; and Doug Hoover, 
Odyssey Research Associates 
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The AAMP5/AAMP-FV Project 


Steven P. Miller 
Collins Commercial Avionics 
Rockwell International 
Cedar Rapids, IA 52498 USA 
spmiller@pobox.cca.rockwell.com 


Mandayam Srivas 
Computer Science Laboratory 
SRI International 
Menlo Park, CA 94025 USA 
srivas@csl.sri.com 


f.b 


Software and digital hardware are increasingly being used in situations where failure could be life threatening, 
such as aircraft, nuclear power plants, weapon systems, and medical instrumentation. Several authors have 
demonstrated the infeasibility of showing that such systems meet ultra-high reliability requirements through 
testing alone [1,2]. Formal methods are a promising approach for increasing our confidence in digital systems, but 
many questions remain on how it can be used effectively in an industrial setting. 

This presentation describes a project, formal verification of the microcode in the AAMP5 microprocessor, 
conducted to explore how formal techniques for specification and verification could be introduced into an industrial 
process. Sponsored by the Systems Validation Branch of NASA Langley and by Collins Commercial Avionics, 
a division of Rockwell International, it was conducted by Collins and the SRI International Computer Science 
Laboratory. The project consisted of specifying in the PVS language developed by SRI [3] a portion of a Rockwell 
proprietary microprocessor, the AAMP5, at both the instruction set and register-transfer levels and using the PVS 
theorem prover to prove the microcode correct for a representative subset of instructions. 

While this presentation includes a brief technical overview (see-{4,5] for a detailed technical discussion), its 
emphasis is on the lessons learned in using PVS for an example of this size and the implications for using formal 
methods in an industrial setting. The central result of this project was to demonstrate the feasibility of formally 
specifying a commercial microprocessor and the use of mechanical proofs of correctness to verify microcode. This 
is particularly significant since the AAMP5 was not designed for formal verification, but to provide a more than 
three fold performance improvement, by pipelining instruction execution, while remaining object code compatible 
with the earlier AAMP2. As a consequence, the AAMP5 is one of the most complex microprocessors to which 
formal methods have been applied. 

Another key result was the discovery of both actual and seeded errors. Two actual microcode errors were 
discovered and corrected during development of the formal specification, illustrating the value of simply creating 
a precise specification. Two seeded errors were systematically uncovered while doing correctness proofs. One of 
these was an actual error that had been discovered after first fabrication but left in the microcode provided to SRI. 
The other error was designed to be unlikely to be detected by walkthroughs, testing, or simulation. 

Several other results emerged during the project, including the ease with which practicing engineers became 
comfortable with PVS, the need for libraries of general purpose theories, the usefulness of formal specification in 
revealing errors, the natural fit between formal specification and inspections, the difficulty of selecting the best style 
of specification for a new problem domain, the high level of assurance provided by proofs of correctness, and the 
need to engineer proof strategies for reuse. 

Many of the costs of the AAMP5 project can be attributed to the overhead of applying an experimental method 
for the first time. To determine how much these costs can be reduced through reuse of the AAMP5 expertise, 
Collins, SRI, and NASA are conducting a follow-on project to verify the microcode in the AAMP-FV, a smaller 
microprocessor design similar to those actually used in autoland systems. A report on the status of this project is 
also presented. 

[1] Butler, R. and G. Finelli, The Infeasibility of Experimental Quantification of Life-Critical Software Reliability, Soft- 
ware Engineering Notes , Vol. 16, No.5, pg. 66-76, December 1991. 

[2] Littlewood, B . and L. Strigini, Validation of Ultra-High Dependability for Software-based Systems, Communications of 
the ACM, Vol. 36, No. 11, pg. 69-80, November 1993. 

[3] Owre, S., J. Rushby, and N. Shankar, PVS: A Prototype Verification System, In Deepak Kapur, Editor, 11th International 
Conference on Automated Deduction, (CADE), pg. 748-752, Saratoga, NY, June 1992, Vol. 607 of Lecture Notes in 
Artificial Intelligence, Springer- Verlag. 

[4] Srivas, M. and S. Miller, Formal Verification of the AAMP5: A Case Study in the Verification of a Commercial Micro- 
processor, to appear in Applications of Formal Methods, Michael G. Hinchey and Jonathan P. Bowen, Editors, Prentice- 
Hall International Series in Computer Science. 

[5] Srivas, M. and S. Miller, Formal Verification of an Avionics Microprocessor, to be submitted as a NASA Contractor Re- 
port. 
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No slides are available 

for 

Joe Prof eta’s portion of this presentation 



Applying Formal Methods to Railway Control 

Doug N. Hoover 

Odyssey Research Associates, Inc. 


In collaboration with Union Switch and Signal, ORA has been carrying on research into 
applying formal methods to system-level railway control modeling and to verification of 
parts of VFRAME++, a railway control CAD system under development by US&S. 

The railway modeling work has produced modeling methods powerful enough to prove 
safety of the conventional block control system. We hope to apply it to newer systems for 
which safety is more problematic. 

The VFRAME++ work centers around correctness of translation from graphical representa- 
tions of circuits to hardware implementing them. Work so far carried out consists of design 
verification of a translation algorithm in PVS. Currently planned is an a posteriori checker 
to show that a particular translation has been done correctly. Such a checker would have 
the advantage of being simple and stand-alone, hence easy to demonstrate to be correct. 
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Outline ofUS&S/ORA Collaboration 
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FM allows fancier programming without added risk. 

O Our translation algorithm was more sophisticated than US\&S's. 
O They were being cautious, we were trying to show off. 






Conclusions 
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Honeywell - Air Transport Systems , HoneyweU - Air Transport Sysloms 
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Formal Tools and Methods for Decision Tables 

Doug N. Hoover 

Odyssey Research Associates , Inc 


This work began with a problem from Honeywell: how to tell whether a specification or 
code involving a complex choice of alternatives is complete (always designates a choice) and 
is consistent (always designates only one choice). It turned out that Honeywell used deci- 
sion tables informally to specify this kind of code. Now, decision tables are a kind of formal 
specification, so we decided that the best solution was to build a specialized tool, TableWise, 
to deal with the problem. 

TableWise checks completeness and consistency of decision tables, as well as generating 
Ada code and documentation from them. It has an original form of structural analysis that 
localizes flaws that prevent decision tables from being complete and consistent. TableWise 
is available from NASA Langley by anonymous ftp (airl6.larc.nasa.gov). A paper on Table- 
Wise will appear in COMPASS ‘95. 

Continuing work related to TableWise includes generating tests from decision tables, 
improving code generation, structuring decision tables to compress information, and mak- 
ing decision tables part of a more all-encompassing specification methodologies. 
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Assertions about decision tables. 
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Session 3: Industry Perspectives on Formal 
Methods 

C. Michael Holloway, Chair 

• Scott French, Loral ATC (No slides available ) 

® Steve Miller, Rockwell-Collins 

• Joe Profeta, Union Switch & Signal (No slides available ) 

® Lance Sherry, Honeywell 
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Session 4: Software Systems (1) 

Ricky W. Butler, Chair 


• Formal Verification for Fault-Tolerant Architectures/PVS Design, by John M. Rushby, SRI 
International 

• Formal Methods Demonstration Project for Space Applications, by John Kelly, Jet Propulsion 
Laboratory; and Ben DiVito, VIGYAN, Inc. 
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Fault-Tolerant Algorithms and the Design of PVS 

John Rushby 
SRI International 


PVS is the most recent in a series of verification systems developed at SRI. Its design was strongly 
influenced, and later refined, by our experiences in developing formal specifications and mechani- 
cally checked verifications for the fault-tolerant architecture, algorithms, and implementations of a 
model “reliable computing platform” (RCP) for life-critical digital flight-control applications. 

Several of the formal specifications and verifications performed in support of RCP are individually 
of considerable complexity and difficulty. But in order to contribute to the overall goal, it has often 
been necessary to modify completed verifications to accomodate changed assumptions or require- 
ments, and people other than the original developer have often needed to understand, review, build 
on, modify, or extract part of an intricate verification. 

In this talk, I will outline the verifications performed, present the lessons learned, and describe 
some of the design decisions taken in PVS to better support these large, difficult, iterative, and col- 
laborative verifications. 

The following article covers this material in more detail: 

“Formal Verification for Fault-Tolerant Architectures: Prolegomena to the Design of PVS” by Sam 
Owre, John Rushby, Natarajan Shankar, and Friedrich von Henke ( IEEE Transactions on Software 
Engineering, Vol 21., No. 2, pp. 107-125). 

This article is available on the World-Wide Web at the following URL: 

http://www.csl.sri.coin/tse95.html 
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Formal methods applied to fault- tolerant algorithms and 
redundancy management for digital flight control 
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Reliable Computing Platform 
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nonfaulty processors can read each other's clocks with error • incrementally refresh state of all processors with 

at most c) majority-voted versions (provides transient recovery) -J 

Enough nonfaulty processors: must collect empirical data on . Optionally, diagnose faulty processors and reconfigure 

individual failure rates, then use (Markov) modeling (increases number of faults that can be tolerated) ^ 




Formal Verifications Performed: Clock Synchronization 
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Formal Verifications Performed: Distributed Consensus 
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"symmetric") and can tolerate more of these (Verified an incorrect variant on the way — bug was masked by 

"Hybrid" algorithms are strictly superior to pure "Byzantine" an in c°rrect axiomatization of "hybrid majority vote") 

ones (e.g., n > 3a + 2s + m rather than n > 3/) 

• Subsequently adapted this algorithm to Draper FTP 

But their analysis is much more complex architecture and verified it in a couple of days 



Clock Synchronization (Again) 
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Allow inexpensive and reliable exploration of alternative designs • Mechanized deductive support in addition to theorem proving 
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Theorem Proving Effectiveness 


Reconciling the Conflicts: Pragmatics 
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May be difficult to believe this, until you’ve experienced the • Allows a lot of the specification to be expressed in the types, so 

personal shock of finding errors in your own specifications that typechecking becomes a very strong check on consistency 



Example: Predicate Subtypes Example: Predicate Subtypes (Higher-Order) 
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Example: Recursive Definition Design Choices in PVS Specification Language 
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Formal Methods Demonstration Project for Space Applications 

Ben L. DiVito 0 n 

' ’ i 

VIGYAN, Inc 


The Space Shuttle program is cooperating in a pilot project to apply formal methods to live require- 
ments analysis activities. As one of the larger ongoing Shuttle Change Requests (CRs), the Global 
Positioning System (GPS) CR involves a significant upgrade to the Shuttle’s navigation capability. 
Shuttles are to be outfitted with GPS receivers and the primary avionics software will be enhanced 
to accept GPS-provided positions and integrate them into navigation calculations. Prior to imple- 
menting the CR, requirements analysts at Loral Space Information Systems, the Shuttle software 
contractor, must scrutinize the CR to identify and resolve any requirements issues. 

We describe an ongoing task of the Formal Methods Demonstration Project for Space Applications 
whose goal is to find an effective way to use formal methods in the GPS CR requirements analysis 
phase. This phase is currently under way and a small team from NASA Langley, ViGYAN Inc. and 
Loral is now engaged in this task. Background on the GPS CR is provided and an overview of the 
hardware/software architecture is presented. We outline the approach being taken to formalize the 
requirements, only a subset of which is being attempted. The approach features the use of the PVS 
specification language to model “principal functions,” which are major units of Shuttle software. 
Conventional state machine techniques form the basis of our approach. 

Given this background, we present interim results based on a snapshot of work in progress. Sam- 
ples of requirements specifications rendered in PVS are offered for illustration. We walk through a 
specification sketch for the principal function known as GPS Receiver State Processing. Results to 
date are summarized and feedback from Loral requirements analysts is highlighted. Preliminary 
data is shown comparing issues detected by the formal methods team versus those detected using 
existing requirements analysis methods. We conclude by discussing our plan to complete the 
remaining activities of this task. 
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PVS Modeling of Principal Functions (Cont'd) Requirements for Receiver State Processing 
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Sample Subfunction of Receiver State Processing Principal Function Interface Types 
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Session 5: Software Systems (2) 

C. Michael Holloway, Chair 


* Ada 9X Language Precision Team, by David Guaspari, Odyssey Research Associates 

• Introduction to Penelope and Its Applications, by David Guaspari , Odyssey Research Associates 
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Formal Methods in the Design of Ada 95 


273/6 


David Guaspari 
Odyssey Research Associates 


l? lO 

f ' 


Formal, mathematical methods are most useful when applied early in the design and implementa- 
tion of a software system— that, at least, is the familiar refrain. I will report on a modest effort to 
apply formal methods at the earliest possible stage, namely, in the design of the Ada 95 program- 
ming language itself. This talk is an "experience report” that provides brief case studies illustrat- 
ing the kinds of problems we worked on, how we approached them, and the extent (if any) to which 
the results proved useful. It also derives some lessons and suggestions for those undertaking future 
projects of this kind 

Ada 95 is the first revision of the standard for the Ada programming language. The revision began 
in 1988, when the Ada Joint Programming Office first asked the Ada Board to recommend a plan 
for revising the Ada standard. The first step in the revision was to solicit criticisms of Ada 83. A 
set of requirements for the new language standard, based on those criticisms, was published in 
1990. A small design team, the Mapping Revision Team (MRT), became exclusively responsible for 
revising the language standard to satisfy those requirements. The MRT, from Intermetrics, is led 
by S. Tucker Taft. 

The work of the MRT was regularly subject to independent review and criticism by a committee of 
Distinguished Reviewers and by several advisory teams — for example, by two User/Implementor 
teams, each consisting of an industrial user (attempting to make significant use of the new lan- 
guage on a realistic application) and a compiler vendor (undertaking, experimentally, to modify its 
current implementation in order to provide the necessary new features). One novel decision estab- 
lished the Language Precision Team (LPT), which investigated language proposals from a mathe- 
matical point of view. 

The LPT applied formal mathematical analysis to help improve the design of Ada 95 (e.g., by clari- 
fying the language proposals) and to help promote its acceptance (e.g., by identifying a verifiable 
subset that would meet the needs of safety-critical applications). The first LPT project, which ran 
from the fall of 1990 until the end of 1992, produced studies of severed language issues: optimiza- 
tion, sharing and storage, tasking and protected records, overload resolution, the floating point 
model, distribution, program errors, and object-oriented programming. The second LPT project, in 
1994, formally modeled the dynamic semantics of a large part of the (almost) fined lemguage defini- 
tion, looking especially for interactions between language features. 
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Applies to one expression at one point in program (environment is 
not a state variable) 





Using the formal model of overload resolution 1 f Results of work on overload resolution 
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end alert_system; 
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Introduction to Penelope 

David Guaspari 
Odyssey Research Associates 
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A formal program verification is a (mathematical) proof that a program executed according to its 
intended model meets some specification. This proves that the algorithm defined by the program is 
correct in the precise technical sense of being consistent with a particular specification. A program 
correct in this sense is free from a large and important class of errors, even though its behavior may 
still produce unintended results — either because the implementation of the progr ammin g language 
itself does not match the model of execution, or because the specification does not correctly express 
the user’s intentions. 

Penelope is a prototype system for interactively developing and verifying programs that are written 
in a rich subset of sequential Ada. Penelope can be used to develop a program and its correctness 
proof incrementally, and in concert with one another. Incrementality is used in a number of ways to 
help make verification more tractable and more productive. For example, if an already-verified pro- 
gram is modified, one can attempt to prove the modified version by replaying and modifying the 
original verification. 

Penelope’s specification language, Larch/Ada, belongs to the family of Larch interface languages. 
Larch/Ada scales up properly, in the sense that it is demonstrably sound to decompose a system 
hierarchically and reason locally about the implementation of each piece. 

Penelope has been applied in various demonstration projects — for specification (guidance control, 
distributed operating system), verification (of off-the-shelf code), and formal development (by non- 
expert as well as expert users). Some features of Penelope have been embodied in AdaWise, a lint- 
like non-interactive tool that warns of the potential for certain dynamic semantic errors in Ada pro- 
grams. 
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Derived predicate transformer semantics can be proven sound for the corre- 
sponding denotational model. 
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Session 6: Hardware Systems 

Paul Miner, Chair 

• The Formal Verification Technology Used on AAMP5, by Mandayam Srivas, SRI International 

• Specification and Verification of VHDL Designs, by Damir Jamsek, Odyssey Research Associates 
® Derivational Reasoning System, by Bhaskar Bose, Derivation Systems Inc. 
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The Formal Verification Used for the AAMP5 and AAMP-FV 


Ms /S' 


It is becoming increasingly evident within the VLSI design industry that the complexity of many current 
hardware designs is outstripping the capability of traditional simulation-based tools to adequately verify 
them. This situation was well-illustrated by the recent floating point bug discovered in Intel’s Pentium pro- 
cessor. The industry is beginning to look at formal verification as a technological alternative to simulation 
for obtaining higher assurance than is currently possible. 




Recently, SRI International and Collins Commercial Avionics, a division of Rockwell International, un- 
dertook a project to explore how formal techniques for specification and verification could be introduced 
into an industrial process. The project, sponsored by the Systems Validation Branch of NASA Langley 
and Collins Commercial Avionics, consisted of specifying in the PVS language a portion of a Rockwell 
proprietary microprocessor, the AAMP5, at both the instruction set and register-transfer levels and using 
the PVS interactive proof-checker to show that the microcode correctly implemented the specified behavior 
for a representative subset of instructions. 


The main goal of the project was two-fold: First, to investigate the feasibility of formally specifying and 
verifying a complex commercial microprocessor that was not expressly designed for formal verification. 
Second, to explore effective ways to transfer the technology to an industrial setting. The choice of the 
AAMP5 satisfied the first goal since the AAMP5 was not designed for formal verification, but to provide 
a more than threefold performance improvement while remaining object-code-compatible with the earlier 
AAMP2, which is used in numerous avionics applications, including the Boeing 737, 747, 757, and 767. 

To satisfy the technology transfer objective, we had to develop a suitable verification methodology and a 
formal infrastructure to make the technology usable by practicing engineers. This infrastructure includes 
techniques for decomposing the microprocessor verification problem into a set of verification conditions 
that the engineers can formulate and strategies to automate the proof of the verification conditions. The 
development of the infrastructure was one of the key accomplishments of the project. Most of the in- 
frastructure and methodology are general enough to be reused for other microprocessors, certainly in the 
verification of another member of the AAMP family. This methodology was used to formally specify the 
entire microarchitecture and more than half of the instruction set and to verify a core set of eleven AAMP5 
instructions representative of several instruction classes. However, the methodology and the formal ma- 
chinery developed are adequate to cover most of the remaining AAMP5 instructions. Although PVS was 
the vehicle of the experiment, the methodology is applicable to other sufficiently powerful theorem provers. 

Another key result of the project was the discovery of both actual and seeded errors. Two actual mi crocode 
errors were discovered during development of the formal specification, illustrating the value of simply cre- 
ating a precise specification. Both were specific to the AAMP5 and were corrected before first fabrication. 
Two additional errors seeded by Collins in the microcode were systematically uncovered by SRI, who knew 
that bugs had been seeded, but not their location or identity, while doing correctness proofs. One of these 
was an actual error that had been discovered by Collins after first fabrication but left in the microcode 
provided to SRI. The other error was designed to be unlikely to be detected by walk-throughs, testing, or 
simulation. 


Steve Miller’s talk earlier in the workshop, gave an overview of the AAMP5 project emphasizing the tech- 
nology transfer process with its administrative and managerial aspects. This talk describes the technical 
approach used in verifying the AAMP5. Please refer to Steve Miller’s slides for the AAMP5 design figures. 
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The Formal Verification Technology Used for the A AMPS 
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normal_macro_machine.next_macro_state(st) = 

st WITH [(dmem) (word2denv(denv(st)) ) (tos(st)+i) 
(pc) := pc(st) + 1, 

(tos) :« tos(st) + 1] 


DPU Environment Assumptions 




ex 

1 

B 

03 


a 

03 

B 

P 



3 

•H 

3 



.3 

1 

ex 



V-/ 

P 

P 


p 


O 

3 


s— ' 


1 

o 


> 


P 



2 

/"V 

•H 

® 


Ed 

3 

3 

x 


O 

T3 

3 

p 


V— ' 

O 

1 



p 

U 

P 

3 


d 

(X 

CO 

O 


1 

O 

Eh 



<D 


•H 

* 


*3 

ac 

P 

P 


O 

o 


3 


o 

OS 

U 

■H 


EX, 

1 


d 


O 

EX 

X~N. 

u 



Ed 

P 

p 


II 



03 



II 

X 

3 


<0 


n 

O 


-o 



u 

X 

o 

p 

A 


a 

A U 


II 


S x 

y-S 

•H P 

U ^ 

d > >- 

> 3 G 

fj *H fli 

HH >- 

Q 
OS 


(X 

►H 

Q 

Ex 

Ex 

»— i 

O 

2 

n 

2 

Ed 

Q 

Ed 


2 

Ed 



o 





rH 


3 

> \ 




-p 

1 

/-s 

® 

w 


p 


p 


-iH 

i—l 

O 





P 

P 

+ 

4-> 


bC 


• 


V-/ 


o 


3 


p 

•* 

bO 

P 



■H 


'w' 

P 

3 


u 


jd 

• * 


w 

•H 

EX 

03 


V 

/-s 

/”*s 

r-s 

.3 

Q 


p 

«P 

P 

>- 

U 




® 


S-/ 

o 

P 

It 

4-> 


p 

A 

O 

s 

® 

<+- 


i 


u 


P 

/—v 

o 


i-4 


2 

® 

1 




d 

<P 

N-r 

B 

i— 1 

P 

*o 


B 


® 

d 

d 


c 

X 

P 

— 

s 

co 

B 

ex 

0) 

o 

O 


d 

l 

fH 

Q 

►H 

3 


co 

co 

O 




/— N 


> 

P 

> 

CO 


H 

1 

> 

I 

E- 

CO 

H 

X 


X 

Ed 

Ed 

Ed 

2 

CO 

2 




Ed 

Ed 

Ed 

CO 


CO 



_2 

Ed 

P 

Ed 


N-/ 



tH 

/«-V 

p 

> 

P 


CO 

•w* 

> 


> 

CO 

2 

H 


Ed 


2 

X 

2 

Ed 

H 

Ed 

X 


X 

H 

/-v 

H 


P 


/N 


/-> 

P 

E3 

P 



w 

o 

X 

Q 


Q 

J 

X 


X 

a 

Ex 

a 


w 


Ex 


Ex 

t-H 

II 

t-H 

II 


II 


p 



+ 

/-s 


P 


+ 

v— r 

+ 


143 


TVlax: AXIOM TVl(t+l) * IF DHLD(t) THEN TVl(t) ELSE TV(t) ENDIF • DLX pipeline [Burch & Dill: 1993] 

• UNITA [Windley: 1994] 



General Microprocessor Correctness 
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Abstraction function must be suitably "skewed” 

Length of instruction cycle can be idefinite not necessarily a 
function of the current visible state 
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Instruction-specific Verification Conditions 
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Is commercial processor verification technically feasible? 
Yes, if carefully planned and executed. 
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DES Specification I DES Specification in Larch (Components) 
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Architecture 1 (Behavior) I Architecture 2 (Clocked) 
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Derivation Systems, Inc. 


N96- 10032 


DRS - Derivational Reasoning System 

Bhaskar Bose 

Derivation Systems, Inc. 

5963 La Place Court, Suite 208 
Carlsbad, California 92008 
Tel: (619) 431-1400 
bose@dvsi.com 

May 11, 1995 

Abstract 

The high reliability requirements for airborne systems requires fault-tolerant archi- 
tectures to address failures in the presence of physical faults, and the elimination of 
design flaws during the specification and validation phase of the design cycle. Although 
much progress has been made in developing methods to address physical faults, design 
flaws remain a serious problem. Formal methods provides a mathematical basis for 
removing design flaws from digital systems. 

DRS (Derivational Reasoning System) is a formal design tool based on advanced 
research in mathematical modeling and formal synthesis. The system implements a ba- 
sic design algebra for synthesizing digital circuit descriptions from high level functional 
specifications. DRS incorporates an executable specification language, a set of correct- 
ness preserving transformations, verification interface, and a logic synthesis interface, 
making it a powerful tool for realizing hardware from abstract specifications. DRS inte- 
grates recent advances in transformational reasoning, automated theorem proving and 
high-level CAD synthesis systems in order to provide enhanced reliability in designs 
with reduced time and cost. 


Copyright (c) 1995 Derivation Systems, Inc. 


All Rights Reserved 
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DRS - Derivational Reasoning System 
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Establish a rigorous framework for reasoning about 
complex designs and guaranteeing integrity in the 
design process. 


Derivation and Verification: 

“Alternate” Modes of Formal Reasoning Integrating Derivation and Verification 
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Stop-and-Copy Garbage Collector [Boyer ‘ 89 ] 
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♦ Algebraic mechanisms for isolating verification 
problems to small building blocks. 
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9( x » y» z ) - tt?(x, 2) -> y, g(dec(x), z, add(y, z)) From this transformed specification, we derive the 

h(x, y, z) = g(x, z, add(y, z)) following structural description: 
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♦ Control and architecture are directly synthesized in 
hardware. 


Derivation methodology, along with the incorporation ♦ Continued development of DRS through Phase II 

of verification and logic synthesis, provides a SBIR Contract, 

powerful tool to support the natural analytical and 

generative reasoning that takes place in engineering ♦ Mechanical proofs of transformation rules, 

practice. 
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Session 7 : Researcher Perspectives on Formal 
Methods 

Jim Caldwell, Chair 

® David Dill, Stanford University (No slides available) 

® Doug Hoover, Odyssey Research Associates 

m Steve Johnson, Indiana University 

• Natarajan Shankar, SRI International (No slides available ) 
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Structural/ Functional Spcciflcation/Dcugn j F\1 j s vcr y p OWCr f u i and should be used for more things than telling people their specs/codc 

• Less :i Black box” nature than FM. 310 wron K- 
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Steve Johnson’s Panel Slides 
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O Real progress in research 
0 Is horizon receding faster? 
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Session 8: Research Issues (1) 

C. Michael Holloway, Chair 


• Safety Analysis, by John C. Knight, University of Virginia 

• Non-standard Analysis and Embedded Software, by Richard Platek, Odyssey Research Associates 
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Safety Analysis 

John C. Knight 

Department of Computer Science, University of Virginia 
Charlottesville, VA 22903 


Case Studies 

We are engaged in a research program in safety-critical 
computing that is based on two case studies. We use these 
case studies to provide application-specific details of the 
various research issues, and as targets for evaluation of 
research ideas. 

The first case study is the Magnetic Stereotaxis System 
(MSS), an investigational device for performing human 
neurosurgery being developed in a joint effort between the 
Department of Physics at the University of Virginia and the 
Department of Neurosurgery at the University of Iowa. 

The system operates by manipulating a small permanent 
magnet (known as a “seed”) within the brain using an exter- 
nally applied magnetic field. By varying the magnitude and 
gradient of the external magnetic field, the seed can be 
moved along a non-linear path and positioned at a site 
requiring therapy, e.g., a tumor. The magnetic field required 
for movement through brain tissue is extremely high, and is 
generated by a set of six superconducting magnets located 
in a housing surrounding the patient’s head. The system 
uses two X-ray cameras positioned at right angles to detect 
in real time the locations of the seed and of X-ray opaque 
markers affixed to the patient’s skull. The X-ray images are 
used to locate the objects of interest in a canonical frame of 
reference. 

The second case study is the University of Virginia 
Research Nuclear Reactor (UVAR). It is a 2 MW thermal, 
concrete-walled pool reactor. The system operates using 20 
to 25 plate-type fuel assemblies placed on a rectangular grid 
plate. There are three scramable safety rods, and one non- 
scramable regulating rod that can be put in automatic mode. 
It was originally constructed in 1959 as a 1 MW system, 
and it was upgraded to 2 MW in 1973. Though only a 
research reactor rather than a power reactor, the issues 
raised are significant and can be related to the problems 
faced by full-scale reactor systems. 

Safety Kernel 

The software in systems like those in our case studies is 
veiy large and complex. We assume that, because of this 
size and complexity, faults will remain in the software for 
an application after development. An approach we are pur- 
suing to deal with this is a software architecture termed a 
safety kernel , a concept directly analogous to the security 
kernel used in security applications. 


f 1 

A security kernel provides assurance that a set of security 
policies is enforced independently of the application pro- 
gram. Verification of the security kernel is sufficient to 
ensure enforcement of those policies encapsulated within 
the security kernel. The application program need not 
enforce the security policies, and it can, in fact, undertake 
actions that would normally lead to violation of the security 
policies with no danger of actual violations taking place. 
The similarity between security concerns and safety con- 
cerns is considerable and the concept of a safety kernel is 
appealing. If the concept were feasible, a safety kernel 
would enforce a set of safety policies by monitoring 
requests to devices, device actions, device status, applica- 
tion software status, and so on. 

We have developed an enforcement safety kernel and inte- 
grated it into our MSS implementation. The safety kernel is 
generated automatically from a formal specification of the 
safety policies, and tests of the MSS instantiation show 
excellent performance. 

Testing 

Systems of this complexity pose significant challenges in 
the area of testing, especially in the large number of possi- 
ble test cases. We are using a technique that we call specifi- 
cation limitation to permit demonstration of useful 
properties by exhaustive testing. By specification limitation 
we mean that the specification for the application is deliber- 
ately limited in several areas to restrict the total number of 
test cases. For example, in the MSS the angles entered by 
the operator for the required direction of motion are 
rounded to 1/10 of a degree. In practice, this is not a signifi- 
cant functional restriction but it permits exhaustive testing 
of the angles used for setting direction. The same approach 
is used with distance. 

A second significant problem in testing complex systems is 
correctness determination, i.e., determining whether the 
outputs are correct. In our MSS implementation, we have 
addressed this problem by the use of reversal checks on the 
entire system. A reversal check computes a program’s input 
from its output and compares this with the actual input. The 
current calculations for the superconducting coils, for 
example, begin with a required force and are very complex. 
Computing the force resulting from the coil currents, how- 
ever, is simple and provides the exact inverse of the current 
calculations. Thus the input can be computed and com- 
pared. A variation on the idea of a reversal check is also 
used by the MSS imaging subsystem. 
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AFETY Kernel Concept \ ( Some Of The MSS Safety Policies 
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Major Safety Kernel Issues A f System Design 
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MSS Testing Architecture \ f MSS Test Case Selection 
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Has No Effect On System Utility 

Permits Significant Statistical Samples If Not Exhaustive 
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Non-Standard Analysis and Embedded Software 

Richard Platek 
Odyssey Research Associates 





richard@oracorp.com 


One model for computing in the future is ubiquitous, embedded computational devices analagous to 
embedded electrical motors. Many of these computers will control physical objects and processes. 
Such hidden computerized environments introduce new safety and correctness concerns whose 
treatment go beyond present Formal Methods. In particular, one has to begin to speak about Real 
Space software in analogy with Real Time software. By this we mean, computerized systems which 
have to meet requirements expressed in the real geometry of space. How to translate such require- 
ments into ordinary software specifications and how to carry out proofs is a major challenge. 

In this talk we propose a research program based on the use of non-standard analysis. Much detail 
remains to be carried out. The purpose of the talk is to inform the Formal Methods co mmuni ty that 
Non-Standard Analysis provides a possible avenue of attack which we believe will be fruitful. 
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"No continuum can not be made up out of indivisibles, as for instance a 
line out of points, granting that the line is continuous and the point indi- 
visible. " Aristotle 
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Basis for an explosive development of new mathematics. 
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Basic Computer Science Question 
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Session 9: Research Issues (2) 

Victor Carreno, Chair 

1 • Hybrid Fault Algorithms, by Pat Lincoln , SRI International 

• Model Checking, by David Dill, Stanford University 
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Hybrid Fault Algorithms 
Patrick Lincoln, SRI International 


Application: Digital Flight Control 


In Association With: 

J. Rushby sri 

N. Suri Allied Signal 
C. Walter (then) Allied Signal 

Sponsored by the NASA Langley Research Center 
under contract NASI 18969 


• Extreme Reliability (10~ 9 system failures/hour) 

• Synchronous Systems 

• NASA RCP 


Byzantine Fault Tolerant Machine Designs 

• SRI’s SIFT (early 80’s) was the first; used 70% 
CPU cycles on overhead for voting etc. 

• Allied- Signal’s MAFT (late 80’s) used hardware as- 
sistance; numerous innovations 

• Draper Lab’s Asymmetric FTP: less hardware 

• Others: TU Vienna MARS, AIPS, FTTP... 


General Points About Verification 

• Informal Proofs are Unreliable: 

In the past, have Found and Corrected Flaws 
in Published Algorithms and Proofs. 

• PVS is an Effective Tool: 

Emphasis on Minimizing Human Effort 

• Proof Reuse Reduces Cost 

• Still Expensive to Prove Anything Interesting 


The Algorithms Used are Complex 
Single Points of Failure 
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General Points About Algorithms 

• Slight Modifications to Traditional Algorithms 

- Can Provide Much Greater Reliability 

- Can Destroy Correctness 

• Hybrid Approach Widely Applicable 

Clock Synchronization 

- Byzantine Agreement 

- Diagnosis 


Byzantine Agreement 

Set of n Generals (Processors) 

m of them are Traitors (faulty) 

All Good Generals Must Agree on Plan of Attack 
All Good Processors Must Agree on Sensor Value 

Make NO Assumptions about Behavior of Traitors 
Make NO Assumptions about Behavior of Faults 


Assume One Commander, Others Are Lieutenants 
Assume One Transmitter, Others Are Receivers 
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The Byzantine Generals Problem: 
Requirements 

Agreement: Nonfaulty Lieutenants Agree on the Value 
Received from the Commander 

Validity: If the Commander is Nonfaulty, the Value 
Received by Nonfaulty Lieutenants is the Value Ac- 
tually Sent 


The Oral Messages Algorithm (OM) 
(Lamport, Shostak, Pease 1982) 

• The Commander Sends Value to Each Lieutenant 

• If r = 0, Each Lieutenant Accepts A Value 
Otherwise, each lieutenant plays the part of the 
general in OM{r - 1) to send value received to other 
lieutenants 


• Each Lieutenant Takes Majority Vote of the Values 
Received 
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Properties of OM 


Hybrid Fault Models 


• For Agreement and Validity Requires n > 3a 
(Best Possible Bound) 

• r 4 * 1 rounds of message exchange 

• Exponential Number of Messages (Moses 93) 

• Need Four Channels to Withstand a Single Fault 

• Formally verified 

(Bevier & Young ’92, Rushby ’92, Shankar ’93) 


• Extreme Positions: 

- Byzantine Approach: Components are Perfect 
or Failed in Unknown Manner 

- FMEA Approach: Components Fail in (many) 
Known Ways; Design Countermeasures for each 
(and combinations) 

• Allied Signal’s Hybrid Fault-Models: 

Include Byzantine Case, Pius a Few Common Cases 
(Fail-Stop, Stuck-At, •*•) 
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Formalizing Hybrid Fault Models; 
Motivation 

• Maximum Assurance 

• Maximum Reliability 

• Minimum Hardware 


Algorithms for Hybrid Fault Models 


• Extend Byzantine Fault- Tolerant Algorithms to 
Deal Better With NonByzantine Faults 


• Survive a Few Bad Faults 
or a Lot of Simple Faults with the Same Protocol 


• Allied Signal’s Reliability Analysis: 
This Provides Superior Reliability 
(McElvany-Hugue ’93) 


li 
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A Hybrid Fault Model 


Requirements for Hybrid Agreement 


• Allied Signal (Thambidurai and Park ’88) 

- (a) Arbitrary (Byzantine, Asymmetric) 

- (s) Symmetric (Value, Stuck- At) 

- (c) Crash (Manifest, Stopped, Out-of-Bounds) 

- ( g ) Good (Nonfaulty, OK) 


Agreement: All nonfaulty receivers agree on the value 
ascribed to the transmitter. 

Validity: If receiver q is nonfaulty, the value it ascribes 
to the transmitter is 

• The correct value if the transmitter is nonfaulty. 

• The value actually sent to all receivers if the 
transmitter is symmetric-faulty. 

• The distinguished value E if the transmitter is 
manifest-faulty. 
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Allied Signal’s Algorithm Z 
(Thambidurai and Park 1988) 

• Like OM 

• When Manifestly Bad Value Received, Record E 

• Lieutenants Pass on E When Receive E 

• Ignore E in majority vote 


Algorithm Z: Flawed 

• n = 5, r = 1, the Commander has a Crash Fault, one 
Lieutenant is a Traitor 

• Good Lieutenants will believe whatever they 
receive from the Traitor: Protocol Fails 

• MAFT Implementation Correct 
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Importance of Validation 

• Lincoln & Rushby Specified Incorrect Algorithm 

• Like Z, But Lieutenants Pass on RE 
“Reported Error” 

• With Incorrect Axiom “HybridJMajority_Vote_l” 

- “Ignore Votes of Crashed Processors” 

- 1-Round Case: OK 

- 2- Round Case: Impossible 

- Should Have Been: “Ignore E Values” 

• With Axiom, “Verified” Incorrect Algorithm 

• Should Validate Axioms With Implementation 


Algorithm OMH 
(Lincoln &: Rushby ’93) 

• Like OM, but 

• Add m + 1 kinds of error value: E, RE, R 2 E etc. 

• If receive R*E, pass on as R irl E 

• Ignore E in majority vote 

• If vote yields R-'E, selected value is R J -1 E 

• Formally Specified and Verified in PVS 
(Lincoln & Rushby ’93) 

Validated Axioms With MJRTY Implementation 


Doubt Anyone Could Get This Right 
Without Formal Verification 
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Algorithm OMH 

Ti — o, T = 1 



Algorithm OMH 

• If r > a, then OMH(r) Tolerates 
n>2a-f-2s + c + r 

If r = a, and s = c = 0, Then n > 3a 
(Same as Traditional Bound) 


• (a) Arbitrary (Byzantine, Asymmetric) 

• (s) Symmetric (Value, Stuck- At) 

• (c) Crash (Manifest, Stopped, Out-of-Bounds) 

• 0) Good (Nonfaulty, OK) 


19 


20 


197 



Fault Combinations Tolerated 


Achieved Reliability: 6 Processor OMH 


Number of Faults | 

Arbitrary 

Symmetric 

Crash 

(byzantine) 

(value) 

(manifest) 

1 

0 

0 

0 

1 

0 

0 

0 

1 


• McElvany-Hugue’s Reliability Analysis 

• Reliability Goal: 10 -9 System Failures/Hour 

- 6 Processor OM: 1.5 x 10 -7 

- 6 Processor OMH: 1.5 x 10" n 


Number of Faults 

Arbitrary 

Symmetric 

Crash 

(byzantine) 

(value) 

(manifest) 

1 

I 

0 

1 

0 

2 

0 

2 

0 

0 

1 

2 

0 

0 
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Formal Specification of OMH in PVS 

uuk[m : nat, n : posuat, T : TYft, error : 7', R, UtiR : ( T — 7']] : Tmloky 
assuming act .ax : ASSUMKriON (V (I : 7'): R(<) # error) 

unactj** : assu MfTiON (V (f : 7 ): UuR(R(0) = 0 

CNOASSUMINU 

rounds: TYrt = upto[mj 

t : VAR T 

feu : l YKt = below In] 
feuset : TYKt = set off feu] 
feu vector : l YKt = [feu — 7‘] 

G\p, ?, 2 : var feu 

: var fcuvector 
caucus : var feuset 
r : var rounds 

iMfURnNO finite .cardinality [feu, *, identity [feu]], 
filters [feu], 

CRrd«set[feu, n, identity [feu] j, 
hybnd_injrty[7‘ f n, error] 

statuses: lYrt — {arbitrary , symmetric, manifest, nonfaulty } 

status : [feu — statuses] 

o(r) : bool = arbitrary(sUtus(;)) 

*(r) : bool - »>mmetric(statu»(;)) 
c(j) : bool = uiAiiifest(stRtus{r)) 
y(s) : bool * uoufaulty( status] r)) 
as( caucus ) : feuset = filter(caucus.u) 
ss(caucus) : feuset = filter^ caucus, a) 
cs( caucus ) : feuset = filter( caucus , c) 
gs(caucus) : feuset = hUer(caucus, g) 

fincArdjill : LtMMA 

I caucus] = |a»(cAucus)| + |ss( caucus)] + ics(cAucus)| + |g*(cAucus)| 

send : [7 , feu, feu — 7'] 

sendl : axiom g ( p ) 3 send(<,p,?) * t 

send 2 : axiom c(p) 2) *eud(f,p,f) = error 

sends : axiom j(p) 2> send(<,p,f) m seud(t,p,r) 

sendJernuiA : LtMMA -> a(y) D send (t,p,q) = send(t,p, ;) 


HMajority2 : ussa 
(V p : p € caucus 3 vi(p) = nj(p)) 

J HMAjority( caucus, Vi) = HMajority (caucus, tty) 

OMH(G',r, i, caucus) : rkcursivk fcuvector = 
if r s 0 

in tN (A p : send(t,G’,p)) 
test (A p : ik p — G* 

tuln seud(f,G’,p) 

test UnR{H Majority (caucus — {G}, 

(A q : OMH(g,r - l,R{send(l,G ' p $)), caucus - (G'})(p)))) 

bNim) 

tNim- 

MLASURt (A G',r,t, caucus -* nat: r) 

Validity_Prop(r) : bool — 

-< a(?) A p € caucus A 7 € caucus 

A | caucus j > 2 X ( I w( caucus)] + [ss{ caucus)]) + |cs(caucus)| + r 
2> OMH(?, r,i ,caucus)(p) = »end(t,f,p) 

Validity ; LtMMA Validity _Prop(r) 

Validity.Final : TMtORtM 

g{p) A -» a(G) A 2 x |o[ +■ 2 x |«[ + [c] + rn < n 
3 OMH(G\ r, t, full*et[fcu))(p) = *eud(f,G\p) 

Agreement_Prop(r) : bool 3 
y(p) A f(«) A p € caucus A caucus A :6 caucus 
A j caucus] > 2 x (|as( caucus) | + |«(caucus)|) + |cs(caucu»)| + r 
A r > |as(caucus)j 

2> OMH(:, r, 1, caucus)(p) = OMH(:, r,t, caucus) (v) 

Agreement : LtMMA Agreements rop(r) 

AgreemeutSinal : mtORtu 

g(p ) A j(<f) A |aj < m A 2 x la] + 2 x |«| + ]e| + m < n 
D OMH(G’ f m, t, fullset[fcu])(p) = OMH(G\m,l,fullset[fcu])( ? ) 
tN u omh 


HM a jority( caucus, v) : T = proj_l( by bridjujrty (caucus, i>, »)) 
HMajority 1 : LtMMA 

|gs( caucus)] > |as(caucq»>] + |ss(caucus)| 

A (V p : p(p) A p € caucus 2 > e(p) = t) 

A t 4 error A (V p : e(p) A p € caucus D e(p) = error) 
2 HMajority( caucus, u) s t 
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Axioms (Assumptions) 


Theorem 1 


sendl : AXIOM g(jp) D send (t,p>q) = t 

send2 : AXIOM c(p) D send(t, p, q) = error 

send3 : AXIOM $(p) D sen d(t,p,q) = send(£, p,z) 

send Jemma : lemma 

a(p) D sen d(t,p, g) — send (t,p, z) 
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Style of Proof 

Bevier &: Young, Rushby, Shankar 

• Inductive Property Definition 

• Induction Property Assertion (Lemma) 

• Theorem in Final Form 

Validity JProp(r) : bool = 

a(q) Ape caucus A q G caucus 
A ]caucus| > 2 x (|as(caucus)| + |ss(caucus)|) 
+ |cs(caucus)| + r 

D OMH(g, r,t,caucus)(p) = send {t,q,p) 

Validity : lemma Validity _Prop(r) 

Validity _Final : theorem 
(g(p) A a(G) 

A 2 x |a| + 2 x |s| + \c\ H- r < n) 

D OMH(G,r,t,fullset[fcu])(p) = send(J, G,p) 
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Validity, Final : THEOREM 
(g{p) A - a (G) 

A 2 x \a\ + 2 x \s\ + |c| + r < n) 
mbox D OMH(G, r,t,fullset[fcu])(p) = send(t, G,p) 


Algorithm OMH(r) satisfies Validity if there are 
more than 2(<z + s) + c + r processors. 

Validity: If The Commander Is not Byzantine- Faulty, 
The Value Received By Nonfaulty Lieutenants Is 
The Value Actually Sent 
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While ReRunning Proof, We Noticed 
Lemma If c = 0 (No Crash Faults), OMH » OM 

OM Is Better Than Earlier Claimed: 

Corollary 1 For any r, Algorithm OM(r) satisfies 
Validity if there are more them 2 (a + s) + r processors. 

Corollary 2 For any r, Algorithm OM(r) satisfies 
Agreement if there sire more than 2(a + s) + r processors 
and r > a. 

OMH = OM + Diagnosis of Manifest Faults 
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More Corollaries 


FTP Architecture 


More Theorems Derived from our New Improved 
Understanding 


Corollary 3: No Good Processors Are Confused 
By Crash Faults: If a = s = 0 (No Byzantine, No Sym- 
metric Faults), OMH is optimal. 


Corollary 4: Error Values (RE, R 2 E, • ■ ) Can Over- 
lap With Real Values. 


Corollary 5: (Slightly Modified) OMH(r) Allows 
Implementations With Only r + 1 Error Values. 
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FTP Architecture 


Interstages 



• Less Total Hardware 

• Only need 3 voters to mask 1 Byzantine fault (!) 

• Technology used in AIPS, FTTP, etc. 
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Algorithm OM-FTP 

• Lala 1986 

• Similar to one-round OM with symmetric voters 

• Formally specified in EHDM 
(Harper, Alger, & Lala ’91) 

No proofs 

• Formally specified in PVS 
(Lincoln &: Rushby ’94) 

Full formal proofs completed in PVS 
Couple of days 
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Hybrid Faults in FTP 

• Integrate OMH and OM-FTP 

• Resist more (simple) faults with less hardware 


Algorithm OMH-FTP 
(Lincoln & Rushby 1994) 

• Combine OMH and OM-FTP 

• Voters pass on £ as RE 

• Interstages pass on E as E 

• Voters use value reflected from interstage, 
NOT original value 


I 
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Formal Verification in PVS 

• Proof Reuse 

• Copy-and-Edit Proof of OM-FTP 

• Grab parts of Proof of OMH 

• M-x Edit Proof 

• Rerunning Edited Proofs 

• Couple of Days Effort 


Copy-and-Edit a Proof 

• Emacs Interface to Edit Proof Commands 

• Kill and Yank 

• Mix-And-Match Partial Proofs 

• Rerun Until Proof Fails 
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Authenticated Agreement 

• Use Cryptographically Secure Signatures 

• Can Tolerate Many More Faults 

However 

• Questionable Cryptographic Assumptions 
(Key Distribution, Complexity Questions) 


Agreement Using Signed Messages (SM) 
(Lamport, Shostak, & Pease 1982) 

• r + 1 Rounds Of Message Exchange 

• If 1 Properly Signed Message At End, Adopt It 
Else General Is Faulty, Adopt Default 

• Requires Only n>r For Agreement and Validity 
(Best Possible Bound) 

• If Signatures Broken, Protocol Fails Horribly 

One Traitor with Key breaks Agreement and Validity for Any Number 
of Rounds, Any Number of Processors 


• Traitor May Simply GUESS Generals Key 
Possible, Though Very Unlikely 


Use Signatures In OMH (OMHA) 
(Gong, Lincoln, Rushby 1995) 


Algorithm OMHA: Signatures Secure 


• Same as OMH, But at Every Step Check Signatures 

• Symmetric, Arbitrary Lieutenants =>■ Manifest 

• Except: Bad Lieutenants Could Send R(E) 

• Worst Case With Secure Signatures Same As OMH 
(Worse Than SM) 

• If Signatures Broken, Same Tolerance as OMH 
(Much Better Than SM) 


It Would Be Nice to Disallow R{E) 
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Algorithm OMHA: Broken Signatures 

• ri = 5,"r = 1, the Commander has a Crash Fault, one 
Lieutenant is a Traitor with Crypto Key 

• Good Lieutenants Not Fooled: 

Protocol Succeeds 
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Algorithm ZA: Broken Signatures 

• n = 5, t = 1 , the Commander has a Crash Fault, one 
Lieutenant is a Traitor with Crypto Key 

• Good Lieutenants will believe whatever they 
receive from the Traitor: Protocol Fails 
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Use Signatures In Z (ZA) 

(Gong, Lincoln, Rushby 1995) 

• Same as Z, But at Every Step Check Signatures 

• Symmetric, Arbitrary Lieutenants =» Manifest 

• Bad Lieutenants Cannot Send R(E) 

• With Secure Signatures, ZA is Optimal 
(Same as SM, Better than OMH) 

• If Signatures Broken, Same Tolerance as Z 
(Better Than SM, Worse Than OMH) 
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Symbolic Fault Injection 
(Gong, Lincoln, Rushby 1995) 

• Using Dill’s Mur<£ Tool (Stanford) 

• Specified OM, OMH, OMHA, SM, Z, ZA 

• Complete, Exhaustive State Exploration 
5 Processors, 4 Values 

• All Above Bounds Correct 
Rediscovered Flaw in Z 
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Symbolic Fault Injection: Clock Synchronization 

Beyond The Worst Case Bound 

• All Above Protocols Require Synchronization 

> 20,000 Fhult Configurations 

(Most Beyond Above Bounds) • Byzantine Processors May Lie About Clock Value 

• Assume Bounded Clock Drift For Good Clocks 

• Goal: Maintain Bounded Clock Skew Within 6 
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Interactive Convergence Algorithm: ICA 
(Lamport &: Melliar-Smith J 85) 

• Broadcast Current Clock Value 

• Compute Egocentric Mean 

- Ignore Values Too Far Off 

n > 3a 

• Algorithm Correct 

• All But One Lemma Incorrect 

• Proof of Main Theorem Incorrect 

• Corrected Versions Formally Verified in EHDM 
(Rushby & VonHenke 93) Couple of Months Effort 

• Young Formally Verified in NQTHM 
Noted Perfect Clocks Excluded By Rushby 
< should be < in Axiom 


Other Clock Synchronization Algorithms 

• Fred Schneider’s General Treatment 

• Natarajan Shankar Formally Specified and Verified 
in EHDM 

Corrected Several Small Flaws 

• Verified That ICA Satisfies Corrected Properties 

• Paul Miner Found Better, Weaker Assumption Set 
Example Lundelius-Lynch Algorithm 
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Hybrid Interactive Convergence: ICAH 
(Rushby 94) 

• Ignore E Values In Egocentric Mean 

n > 3a + 2s + c 

• Extend Hybrid Fault Model to Link Faults: 

- Model Links Separate From Processors 

- Links Only Introduce Manifestly Bad Values 

- Two Links Between Each Pair 
(In, Out) 

- Asymmetric: Good or Manifest Value 

n > 3a + 25 + c + / 

• Formally Specified and Verified in EHDM 
(Rushby 94) 

Couple of Days Effort 


Diagnosis 

• Key Part of FDIR: 

Detection, Identification, and Reconfiguration 

• Off-Line: After/Between Missions 

- Requires Record Keeping/Testing Harnesses 

- Few Additional Mechanisms In-Flight 

- May Miss Intermittent Faults 
-Use Test Injection Sequences 

• On-Line: During Mission 

- Requires Additional Safety- Critical Mechanisms 

- Consumes Mission Resources 

— Survives More Faults During Mission 
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Benefits of On-Line Diagnosis 


OMH Masks Crashes Already: Why Diagnose? 


On-Line Diagnosis 

• Easy Without Byzantine Faults 

• All Faults Are Symmetric 
Private Accusations Correct 


• Diagnose ALL Symmetric Faults 
Greater Reliability 


• Impossible To Perfectly Diagnose Byzantine Faults 


• Diagnose SOME Byzantine Faults 
Greater Reliability 

• Manifest Faults May Become Worse 


• Stop Progression: 

Manifest =>• Symmetric =*■ Byzantine 

• Restart Crashed Processor 
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Desirable Properties 


On-Line Byzantine Diagnosis 

• PMC, Comparison Testing Inappropriate 


Correctness (Fairness, Soundness): • Shin & Ramanat han ’87 (Requires Authentication) 

Every Processor Diagnosed Faulty IS Faulty 

• Ramarao & Adams ’88 (Optimal, NP-hard) 
Extreme Cost 

Completeness: 

Every Faulty Processor is Diagnosed Faulty . Walter ’90, Walter, Suri & Hugue ’94: Spectrum 

of Algorithms 
PP, PLP, DD, HD 
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Algorithm PP: Walter ’90 

• Assume Symmetric Faults 

• Correctness and Completeness 

• Formally Specified and Verified in PVS 
(Lincoln 95) 

Couple Weeks Effort 


Algorithm PP 

• Monitor All Incoming Messages 

• For Incorrect Messages, Record Accusation 

• At End of Period, Broadcast Accusations 

• If > \N/2] Accusations, Declare Faulty 

- Accuse All Non- Accusing Processors 

• If < \N/2] Accusations, Declare NonFaulty 

- Accuse All Accusing Processors 

• Iterate 
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Algorithm PLP 


Algorithm PLP: Walter ’90 

PP with Link Faults 

Correctness and Partial Completeness 

Formally Specified and Verified in PVS 
(Lincoln 95) 

Heavy Reuse of Lemmas/Proofs from PP 
Couple Days Effort 


• Like PP 

• Except, two kinds of Error: 

Symmetric / Asymmetric 

• Based on Kind of Erroneous Message 

• Asymmetric Errors Assumed to Be Manifest 
as Asymmetric-Looking Values 


58 


Algorithm DD: Walter, Suri, & Hugue ’94 

• True Byzantine Fault Diagnosis 

— Local Detection 

- Byzantine Agreement on Detection: OMH 

- Combine into Diagnosis 

• Correctness and Partial Completeness 

• Formally Specified and Verified in PVS 
(Lincoln 95) 


Algorithm DD 

• Monitor All Incoming Messages 

• For Incorrect Messages, Record Accusation 

• Use OMH to Agree on Accusations 

• If > |7V/2] Accusations, Declare Faulty 

• If < \N/ 2] Accusations, Declare NonFaulty 

• Iterate 


• Heavy Reuse of Lemmas/Proofs 
from OMH & PLP 
Couple Hours Effort 
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Algorithm HD: Walter, Suri, & Hugue ’90 

• True Byzantine Fault Diagnosis 


• Like DD 


Algorithm HD 


• Except, Different Penalty Weights for 

• DD with Penalties, Accumulation, Decay Different Errors 

• Decay of Accusations 

. Correctness and Partial Completeness . Cumulative Penalty > K, Declare Faulty 

• Formally Specified and Verified in PVS 
(Special Case) (Lincoln 95) 

• Heavy Reuse of Lemmas /Proofs From DD 
Couple Hours Effort 


Benefits of Formal Verification 

• Identify Exact Assumptions 

• Explore Alternatives 

• Extremely High Assurance 

- Diagnosis Is Safety- Critical 


Important Properties of User 

• Mathematical Sophistication 

• Application-Area Knowledge 

• Experience 


• Documentation 
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Important Properties of Proof Checker 

• Cite Previous Lemmas and Theorems 

• Library Mechanism 

• Reuse Parts of Proofs 

• Fast Cycle Time (Hu man- Computer-Human) 

• Understandable Output in the Loop 

• Heavy Controllable Automation 
(Rewriting, Decision Procedures) 

• Exploration Tools 
(Mur0) 


Bottom Line 

Exploration of Algorithms, Assumptions 

• Assurance: Formal Verification 

• Little Hardware: FTP 

• Cheap Reliability: Hybrid Algorithms 

- Byzantine Case Ensures Coverage 

- Simple Cases Handled Cheaper 

- More Reliability With Same Hardware 

- Widely Applicable Paradigm 
Clocks, Agreement, Diagnosis 
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i 

i 

“The virtue of a logical proof is not that it compels 
I belief but that it suggests doubts” (Lakatos) 

| Summary 

| • Good Hybrid Fault- Tolerant Algorithms 

• Informal Proofs are Unreliable 

• Real Theorem Proving Still Expensive 

• Interesting Examples Now Feasible 

• Human-Computer Interaction A Key Element 

• Proof Reuse Lowers Cost of Formal Verification 
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Model Checking 

David L. Dill 
Stanford University 


N96- 10035 

/£ <o 


Formal methods have not had the kind of impact we might have hoped. I suggest that the reason is 
economic: the cost/benefit ratio is unacceptable in many cases, and unproven in most. Hence, 
research should examine the question of reducing costs, in the form of labor and time. 


Automatic formal verification methods for finite-state systems, also known as model-checking, suc- 
cessfully reduce labor costs since they are mostly automatic. Model checkers explicitly or implicitly 
enumerate the reachable state space of a system, whose behavior is described implicitly, perhaps by 
a program or a collection of finite automata. Simple properties, such as mutual exclusion or 
absence of deadlock, can be checked by inspecting individual states. More complex properties, such 
as lack of starvation, require search for cycles in the state graph with particular properties. 

Specifications to be checked may consist of built-in properties, such as deadlock or “unspecified 
receptions” of messages, another program or implicit description, to be compared with a simulation, 
bisimulation, or language inclusion relation, or an assertion in one of several temporal logics. 
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There are many success stories of verification tools finding bugs in protocols or hardware control- 
lers. In some cases, these tools have been incorporated into design methodology. 


Research in finite-state verification has been advancing rapidly, and is showing no signs of slowing 
down. Recent results include probabilistic algorithms for verification, exploitation of symmetry and 
independent events, and the use symbolic representations for Boolean functions and systems of lin- 
ear inequalities. 


One of the most exciting areas for further research is the combination of model-checking with theo- 
rem-proving methods. I will briefly describe some initial forays into this area. 


pass m mr fe&st; 
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• a table of states already searched 
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All of these systems are currently in use in industry. * Methods supporting abstraction and refinement 
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Session 10: Research Issues (3) 

Ricky W. Butler, Chair 

* The DDD Scheme Machine, by Steve Johnson , Indiana University 

• Formal Development of a Clock Synchronization Circuit, by Paul Miner , NASA Langley 
Research Center 
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The Scheme Machine: ^ ^ ^ 

A Case Study in Progress in Design Derivation at System Levels 

Steven D. Johnson p ^ 

Indiana University 


The Scheme Machine is one of several design projects of the Digital Design Derivation group at 
Indiana University. It differs from the other projects in its focus on issues of system design and its 
connection to surrounding research in programming language semantics, compiler construction, 
and programming methodology underway at Indiana and elsewhere. The genesis of the project 
dates to the early 1980s, when digital design derivation research branched from the surrounding 
research effort in programming languages. Both branches have continued to develop in parallel, 
with this particular project serving as a bridge. However, by 1990 there remained little real inter- 
action between the branches and recently we have undertaken to reintegrate them. 

On the software side, researchers have refined a mathematically rigorous (but not mechanized) 
treatment starting with the fully abstract semantic definition of Scheme and resulting in an effi- 
cient implementation consisting of a compiler and virtual machine model, the latter typically real- 
ized with a general purpose microprocessor. The derivation includes a number of sophisticated 
factorizations and representations and is also deep example of the underlying engineering method- 
ology. 

The hardware research has created a mechanized algebra supporting the tedious and massive 
transformations often seen at lower levels of design. This work has progressed to the point that 
large scale devices, such as processors, can be derived from first-order finite state machine specifi- 
cations. This is roughly where the language oriented research stops; thus, together, the two efforts 
establish a thread from the highest levels of abstract specification to detailed digital implementa- 
tion. 

The Scheme Machine project challenges hardware derivation research in several ways, although 
the individual components of the system are of a similar scale to those we have worked with before. 
The machine has a custom dual-ported memory to support garbage collection. It consists of four 
tightly coupled processes — processor, collector, allocator, memory — with a very non-trivial synchro- 
nization relationship. Finally, there are deep issues of representation for the run-time objects of a 
symbolic processing language. 

The research centers on verification through integrated formal reasoning systems, but is also 
involved with modeling and prototyping environments. Since the derivation algebra is based on an 
executable modeling language, there is opportunity to incorporate design animation in the design 
process. We are looking for ways to move smoothly and incrementally from executable specifications 
into hardware realization. For example, we can run the garbage collector specification, a Scheme 
program, directly against the physical memory prototype, and similarly, the instruction processor 
model against the heap implementation. 
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Context, Motivation, and Goals of the Research 
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Scheme Machine Prototyping Environment Scheme Machine Architecture 
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Incremental Construction of Hardware Incremental Construction of Hardware 
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N96- 10037 

/ 

Formal Development of a Clock Synchronization Circuit 

Paul S. Miner 


This talk presents the latest stage in a formal development of a fault-tolerant clock synchronization 
circuit. The development spans from a high level specification of the required properties to a circuit realizing 
the core function of the system. 

An abstract description of an algorithm has been verified to satisfy the high-level properties using the 
mechanical verification system Ehdm [2]. This abstract description is recast as a behavioral specification 
input to the Digital Design Derivation system (DDD) developed at Indiana University [1], DDD provides 
a formal design algebra for developing correct digital hardware. Using DDD as the principle design envi- 
ronment, a core circuit implementing the clock synchronization algorithm was developed [3]. The design 
process consisted of standard DDD transformations augmented with an ad hoc refinement justified using the 
Prototype Verification System (PVS) from SRI International [4]. 

Subsequent to the above development, Wilfredo Torres- Pomales discovered an area-efficient realization 
of the same function [5]. Establishing correctness of this optimization requires reasoning in arithmetic, so a 
general verification is outside the domain of both DDD transformations and model-checking techniques. 

DDD represents digital hardware by systems of mutually recursive stream equations. A collection of PVS 
theories was developed to aid in reasoning about DDD-style streams. These theories include a combinator 
for defining streams that satisfy stream equations, and a means for proving stream equivalence by exhibiting 
a stream bisimulation. 

DDD was used to isolate the sub-system involved in Torres-Pom ales’ optimization. The equivalence 
between the original design and the optimized verified was verified in PVS by exhibiting a suitable bisimu- 
lation. The verification depended upon type constraints on the input streams and made extensive use of the 
PVS type system. The dependent types in PVS provided a useful mechanism for defining an appropriate 
bisimulation. 


References 

[1] Bhaskar Bose. DDD - A Transformation System for Digital Design Derivation. Technical Report 331, 
Computer Science Dept. Indiana University, May 1991. 

[2] Paul S. Miner. Verification of fault- tolerant clock synchronization systems. Technical Paper 3349, NASA, 
Langley Research Center, Hampton, VA, November 1993. 

[3] Paul S. Miner, Shyamsundar Pullela, and Steven D. Johnson. Interaction of formal design systems in the 
development of a fault- tolerant clock synchronization circuit. In Proceedings 13th Symposium on Reliable 
Distributed Systems , pages 128-137, Dana Point, CA, October 1994. 

[4] Sam Owre, John Rushby, Natarajan Shankar, and Friedrich von Henke. Formal verification for fault- 
tolerant architectures: Prolegomena to the design of PVS. IEEE Transactions on Software Engineering , 
21(2) : 107— 125, February 1995. 

[5] Wilfredo Torres- Pomales. An optimized implementation of a fault-tolerant clock synchronization circuit. 
Technical Memorandum 109176, NASA, Langley Research Center, Hampton, VA, February 1995. 


225 





Design Hierarchy — New Informal Description of Algorithm 
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Torres-Pomales’ Optimization Signal Assumptions Justifying Optimization 




nth (S : Stream , n : nat) : alpha «• hd(iterate(tl,n)(S)) 
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co_induct : THEOREM (EXISTS (R: Bisimulation): R(X, Y)) 



Stream Equations for Optimized Sub-Circuit PVS Definitions for Circuit Verification 
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C{R)-. TYPE = 

{II Invariant(NOT R => EQ(tl(I) ,INC(I)))} 



Correctness Theorem Proof of Optimize.correct by co-induction 
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! Appendix B: 

Comments from Attendees on the Workshop 


• That was one of the best conferences I’ve attended. Not only were most of the talks good but there 
was a particularly good mix of people at the workshop who were eager to talk about their work 
and related ideas. 

| • This workshop was much more valuable than FM Europe, at a fraction of the cost. 

i • This was the most well organized workshop I’ve ever attended at Langley. 

• Thanks for the conference. I picked up a lot of useful information. 

• There is no regular meeting for the formal methods community. How about expanding your 
meetings to be yearly with an agenda set up like the GSFC SEL? Just thought I would ask. 

• ... thanks again for this year’s meeting. It was great fun. 

• You all did a super job! Keep up the good work. 

j • I thoroughly enjoyed the chance to interact with people at FMWS95. 

• When will the proceedings be ready? I’m looking forward to studying them. 

| 

i 
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Appendix C: 

Detailed Overview of NASA Langley’s Formal Methods Program 


This paper is an expanded version of a paper presented at COMPASS 95. 
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Abstract 

This paper presents an overview of NASA Langley’s research program in formal methods. The major 
goals of this work are to make formal methods practical for use on life critical systems, and to orchestrate 
the transfer of this technology to U.S. industry through use of carefully designed demonstration projects. 
Several direct technology transfer efforts have been initiated that apply formal methods to critical sub- 
systems of real aerospace computer systems. The research team consists of five NASA civil servants and 
contractors from Odyssey Research Associates, SRI International, and Vi'GYAN Inc. 




<V'; 


PAGE„ 


. INTENTIONALLY BLANK 


247 



1 Rationale For a Formal Methods Research Program 

NASA Langley Research Center has been developing techniques for the design and validation of flight critical 
systems for over two decades. Although much progress has been made in developing methods to accommodate 
physical failures, design flaws remain a serious problem [52, 67, 35, 1, 44, 30, 74]. 

A 1991 report by the National Center For Advanced Technologies 1 identified “Provably Correct System 
Specification” and “Verification Formalism For Error- Free Specification” as key areas of research for future 
avionics software and ultrareliable electronics systems [2]. Aerospace engineers who attended the NASA- 
LaRC Flight Critical Digital Systems Technology Workshop listed techniques for the validation of concurrent 
and fault-tolerant computer systems high on the list of research priorities for NASA [58]. 

1.1 Why Formal Methods Is Necessary 

Digital systems (both hardware and software) are notorious for their unpredictable and unreliable behavior: 

Studies have shown that for every six new large-scale software systems that are put into operation, 
two others are cancelled. The average software development project overshoots its schedule by 
half; larger projects generally do worse. And three quarters of all large systems are “operating 
failures” that either do not function as intended or are not used at all. 

Despite 50 years of progress, the software industry remains years-perhaps decades-short of the 
mature engineering discipline needed to meet the demands of an informat ion- age society [31]. 

Lauren Ruth Wiener describes the software problem in her book, Digital Woes: Why We Should Not Depend 
Upon Software : 

Software products — even programs of modest size — are among the most complex artifacts that 
humans produce, and software development projects are among our most complex undertakings. 

They soak up however much time or money, however many people we throw at them. 

The results are only modestly reliable. Even after the most thorough and rigorous testing some 
bugs remain. We can never test all threads through the system with all possible inputs[95]. 

The hardware industry also faces serious difficulties, as evidenced by the recent design error in the Pentium 
floating-point unit. In response to an outcry over the design flaw in the Pentium floating point unit, Intel’s 
President, Andy Grove, wrote on the comp.sys.intel Internet bulletin board: 

After almost 25 years in the microprocessor business, I have come to the conclusion that no 
microprocessor is ever perfect; they just come closer to perfection with each stepping. In the life 
of a typical microprocessor, we go thru [sic] half a dozen or more such steppings.... 

In a recent Washington Post article, Michael Schrage wrote: 

Pentium type problems will prove to be the rule — rather than the isolated, aberrant exceptions — 
as new generations of complex hardware and software hit the market. More insidious errors and 
harmful bugs are inevitable. That is the new reality [82]. 

For life critical systems, errors may mean disaster. The potential for errors is high, because these systems 
must not only perform their functions correctly, but also must be able to recover from the effects of failing 
components (in order to meet stringent ultrareliability requirements.) Often the physical fault-tolerance 
features of these systems are more complex and susceptible to design error than any of the basic functions 
of the system. John Rushby writes: 

Organization of redundancy and fault-tolerance for ultra-high reliability is a challenging problem: 
redundancy management can account for half the software in a flight control system and, if less 
than perfect can itself become the primary source of system failure [70]. 

1 A technical council funded by the Aerospace Industries Association of America (AIA) that represents the major U.S. 
aerospace companies engaged in the research, development and manufacture of aircraft, missiles and space systems, and related 
propulsion, guidance, control and other equipment. 


In a comprehensive assessment of formal methods [77], John Rushby discusses several notorious examples of 
such failures. These include the following: 

• The asynchronous operation of the AFTI-F16 and sensor noise led each channel to declare the other 
channels failed in flight test 44. The plane was flown home on a single channel. Other potentially 
disastrous bugs were detected in flight tests 15 and 36. 

• The HiMAT crash landed without its landing gear due to a design flaw. The problem was traced to a 
timing change in the software that had survived extensive testing. 

• A bug in the YC-14 redundancy management was found during flight test. The bug caused a large 
mistracking between redundant channels. 

• In flight tests of the X31, the control system went into a reversionary mode four times in the first nine 
flights, usually due to a disagreement between the two air data sources. 

• The nationwide saturation of the AT&T switching systems on January 15, 1990 was caused by a timing 
problem in a fault-recovery mechanism. 

• The first Shuttle mission (STS-1) was scrubbed because the fifth backup computer could not be syn- 
chronized with the other four. 

Three basic strategies are advocated for dealing with the design fault problem for the life-critical system: 

1. Testing (Lots of it) 

2. Design Diversity (i.e. software fault tolerance: N-version programming, recovery blocks, etc.) 

3. Fault Avoidance (i.e. formal specification/verification, automatic program synthesis, reusable modules) 

The problem with life testing is that in order to measure ultrareliability one must test for exorbitant 
amounts of time. For example, to measure a 10“ 9 probability of failure for a 1 hour mission one must test 
for more than 10 9 hours (114,000 years). 

The basic idea is to use separate design and implementation teams to produce multiple versions from the 
same specification. At runtime, non-exact threshold voters are used to mask the effect of a design error in 
one of the versions. The hope is that the design flaws will manifest errors independently or nearly so. By 
assuming independence, one can obtain ultrareliable-level estimates of reliability, even with failure rates for 
the individual versions on the order of 10~ 4 /hour. Unfortunately, the independence assumption has been 
rejected at the 99% confidence level in several experiments for low reliability software [47, 48]. 

Furthermore, the independence assumption cannot be validated for high reliability software because of the 
exorbitant test times required. If one cannot assume independence then one must measure correlations. This 
is infeasible as well; it requires as much testing time as life-testing the system, because the correlations must 
be in the ultrareliable region in order for the system to be ultrareliable. Therefore, it is not possible, within 
feasible amounts of testing time, to establish that design diversity achieves ultrareliability. Consequently, 
design diversity can create an “illusion” of ultrareliability without actually providing it. For a more detailed 
discussion, see [15, 14]. 

We believe that formal methods offer the only intellectually defensible method for handling design faults. 
Since the often quoted 1 - 10“ 9 reliability is clearly beyond the range of quantification, we have no choice but 
to develop life critical systems in the most rigorous manner available to us, which is use of formal methods. 

1.2 What is Formal Methods 

Engineering relies heavily on mathematical models and calculation to make judgments about designs. This 
is in stark contrast to the way in which software systems are designed — with ad hoc technique and after- 
implementation testing. Formal methods bring to software and hardware design the same advantages that 
other engineering endeavors have exploited: mathematical analysis based on models. Formal methods are 
used to specify and model the behavior of a system and to formally verify that the system design and 
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implementation satisfy functional and safety properties. In principle, these techniques can produce error- 
free design; however, this requires a complete verification from the requirements down to the implementation, 
which is rarely done in practice. 

Thus, formal methods is the applied mathematics of computer systems engineering. It serves a similar role 
in computer design as Computational Fluid Dynamics (CFD) plays in aeronautical design, providing a means 
of calculating and hence predicting what the behavior of a digital system will be prior to its implementation. 

The tremendous scientific potential of formal methods has been recognized by theoreticians for a long 
time, but the formal techniques have remained the province of a few academicians, with only a few exceptions 
such as the Transputer [3] and the IBM CICS project [38]. The first five years of NASA Langley’s program 
have advanced the capabilities of formal methods to the point where commercial exploitation is near. 

There are many different types of formal methods with various degrees of rigor. The following is a useful 
(first-order) taxonomy of the degrees of rigor in formal methods: 

Level-1: Formal specification of all or part of the system. 

Level-2 : Formal specification at two or more levels of abstraction and paper and pencil proofs 
that the detailed specification implies the more abstract specification. 

Level-3: Formal proofs checked by a mechanical theorem prover. 

Level 1 represents the use of mathematical logic, or a specification language that has a formal semantics, 
to specify the system. This can be done at several levels of abstraction. For example, one level might 
enumerate the required abstract properties of the system, while another level describes an implementation 
that is algorithmic in style. 

Level 2 formal methods goes beyond Level 1 by developing pencil-and-paper proofs that the concrete 
levels logically imply the abstract, property-oriented levels. Level 3 is the most rigorous application of 
formal methods. Here one uses a semi-automatic theorem prover to make sure that all of the proofs are 
valid. The Level 3 process of convincing a mechanical prover is really a process of developing an argument 
for an ultimate skeptic who must be shown every detail. 

It is important to realize that formal methods is not an all-or-nothing approach. The application of formal 
methods to the most critical portions of a system is a pragmatic and useful strategy. Although a complete 
formal verification of a large complex system is impractical at this time, a great increase in confidence in the 
system can be obtained by the use of formal methods at key locations in the system. For more information 
on the basic principles of formal methods, see [16]. 

2 Goals of Our Program, Strategy, and Research Team 

The major goals of the NASA Langley research program are to make formal methods practical for use 
on life critical systems developed in the United States, and to orchestrate the transfer of this technology 
to industry through use of carefully designed demonstration projects. Our intention is to concentrate our 
research efforts on the technically challenging areas of digital flight- control systems design that are currently 
beyond the state-of-the-art, while initiating demonstration projects in problem domains where current formal 
methods are adequate. The challenge of the demonstration projects should not be underestimated. That 
which is feasible for experts that have developed the tools and methods is often difficult for practitioners in 
the aerospace industry. There is often a long “learning curve” associated with the tools, the tools are not 
production-quality, and the tools have few or no examples for specific problem domains. Therefore, we are 
setting up cooperative efforts between industry and the developers of the formal methods to facilitate the 
technology transfer process. 

This strategy leverages the huge investment of ARPA and the National Security Agency in development 
of tools and concentrates on the problems specific to the aerospace problem domain. NASA Langley has 
not sponsored the development of any general-purpose theorem provers. However, the technology transfer 
projects have lead to significant improvements in the Prototype Verification System (PVS) theorem prover[70] 
that SRI International (SRI) is developing. Several domain-specific tools are being sponsored: (1) Tablewise, 
(2) VHDL-analysis tool, and (3) DRS. These tools are discussed in later sections. 

It is also important to realize that formal methods include a large class of mathematical techniques and 
tools. Methods appropriate for one problem domain may be totally inappropriate for other problem domains. 


The following are some of the specific domains in which our program has concentrated: (1) architectural-level 
fault tolerance, (2) clock-synchronization, (3) interactive consistency, (4) design of hardware devices such as 
microprocessors, memory management units, DMA controllers, (5) asynchronous communication protocols, 
(6) design and verification of application-specific integrated circuits (ASICS), (7) Space Shuttle software, (8) 
navigation software, (9) decision tables, (10) railroad signaling systems. 

We are also interested in applying formal methods to many different portions of the life-cycle, such as 
(1) requirements analysis, (2) high-level design, (3) detailed design, and (4) implementation. 

Often, there is a sizable effort associated with the development of the background mathematical theories 
needed for a particular problem domain. Although such theories are reusable and in the long run can become 
“cost-effective”, the initial costs can be a deterrent for industry. Therefore, one of the goals of the NASA 
Langley program is to build a large body of background theories needed for aerospace applications. 

We also have been involved with standards activities in order to strengthen the United States commitment 
to safety. 

2.1 Technology Transfer 

The key to successful technology transfer is building a cooperative partnership with a customer. In order 
for this partnership to work, NASA Langley must become directly involved in specific problem domains 
of the aerospace industry 2 . NASA must also effectively communicate its basic research accomplishments 
in a manner that reveals a significant potential benefit to the aerospace community. Equally important 
is the need for industry to make an investment to work together with NASA on joint projects to devise 
demonstration projects that are realistic and practical. The ultimate goal of our technology transfer process 
is for formal methods to become the “state-of-the-practice” for U.S. industry development of ultrareliable 
digital avionics systems. However, before we can develop new tools and techniques suitable for adoption by 
industry, we must work with the system developers in industry to understand their needs. We must also 
overcome the natural skepticism that industry has of any new technology. 

Our basic approach to technology transfer is as follows. The first step is to find an industry representative 
who has become interested in formal methods, believes that there is a potential benefit of such methods, 
and is willing to work with us. The next step is to fund our formal methods research team to apply formal 
methods to an appropriate example application. This process allows the industry representative to see 
what formal methods are and what they have to offer, and it allows us (the formal methods team) to learn 
the design and implementation details of state-of-the-practice components so we can better tailor our tools 
and techniques to industry’s needs. If the demonstration project reveals a significant potential benefit, the 
next stage of the technology transfer process is for the industry representative to initiate an internal formal 
methods program, and begin a true cooperative partnership with us. 

Another important part of our technology transfer strategy is working with the Federal Aviation Ad- 
ministration (FAA) to update certification technology with respect to formal methods. If the certification 
process can be redefined in a manner that awards credit for the use of formal methods, a significant step 
towards the transfer of this technology to the commercial aircraft industry will have been accomplished. 

Langley has also been sponsoring a series of workshops on formal methods. The first workshop, held in Au- 
gust 1990, focused on building cooperation and communication between U.S. formal methods researchers [18]. 
The second, held in August 1992, focused on education of the U.S. aerospace industry about formal methods[43] 
A third workshop will be held in May 1995. 

Another component of our technology transfer strategy, is to use the NASA’s Small Business Innova- 
tive Research (SBIR) program to assist small businesses in the development of commercially viable formal 
methods tools and techniques. The first contracts under the program began in early 1994. 

Finally, to facilitate technology transfer, much information on NASA Langley’s formal methods research 
is available on the Internet via either anonymous FTP or World Wide Web. PostScript and DVI versions 
of many research papers are available through anonymous FTP on machine deduction.laxc.nasa.gov (IP 
address: 128.155. 18.16) in directory pub/fm. This directory, and much more information, is also available 
through World Wide Web, using the following Uniform Resource Locator: 

2 To date, our efforts have concentrated on the aerospace industry, but we are actively seeking partners from other industries 
also. 



http://atb-www.larc.nasa.gov/fm.html 

2.2 FAA/RTCA Involvement 

As the federal agency responsible for certification of civil air transports, the FAA shares our interest in 
promising approaches to engineering and validating ultrareliable flight-control systems. Additionally, be- 
cause the FAA must approve any new methodologies for developing life-critical digital systems for civil air 
transports, their acceptance of formal methods is a necessary precursor to its adoption by industry system 
designers. We are working with Pete Saraceni of the FAA Technical Center and Mike DeWalt, FAA National 
Resource Specialist for Software, to insure that our program is relevant to the certification process. The 
FAA has co-sponsored some of our work. John Rushby of SRI gave a tutorial on formal methods at an FAA 
Software Advisory Team (SWAT) meeting at their request. The SWAT team suggested that we include an 
assessment of formal methods in an ongoing Guidance Control Software (GCS) experiment in our branch; 
Odyssey Research Associates (ORA) developed a formal specification of the GCS application. 

John Rushby has written a chapter for the FAA Digital Systems Validation Handbook Volume III on 
formal methods[20]. The handbook provides detailed information about digital system design and validation 
and is used by the FAA certifiers. In preparation for this chapter, Rushby produced a comprehensive analysis 
of formal methods [77]. 

George Finelli, the former assistant Branch Head of the System Validation Methods Branch (the Branch 
in which the formal methods team worked before NASA Langley’s reorganization in 1994) and a member 
of the RTCA committee formed to develop DO-178B, together with Ben Di Vito (ViGYAN Inc.), was 
instrumental in including formal methods as an alternate means of compliance in the DO-178B standard. 

Currently, members of the Langley staff are involved in RTCA committees SC- 180 (Airborne Electronic 
Hardware) and SC- 182 (Minimal Operating Performance Standard for an Airborne Computer Resource). 

2.3 Team 

The Langley formal methods program involves both local researchers and industrial / academic researchers 
working under contract to NASA Langley. Currently the local team consists of five civil servants and one 
contractor (ViGYAN Inc.). The lead NASA Langley formal methods researcher, Ricky W. Butler, may be 
contacted through electronic mail to R. W. Butler ©LaRC .NASA.GOV. 

NASA Langley has recently awarded two five-year task-assignment contracts specifically devoted to formal 
methods (from the competitive NASA RFP 1-132-DIC.1021). The selected contractors were SRI Interna- 
tional (SRI) and Odyssey Research Associates (ORA). This was a follow-on contract from the previous 
competitive contract that had awarded three contracts to SRI, ORA, and Computational Logic Inc. (CLI). 

3 Current Technology Development and Transfer Projects 

3.1 AAMP5 / A AMP-FV Project 

In 1993, NASA Langley initiated a joint project involving Collins Commercial Avionics and SRI International. 
The goal was to investigate the application of formal techniques to a commercial microprocessor design, 
the Collins AAMP5 microprocessor. The AAMP5 is the latest member of the CAPS/AAMP family of 
microprocessors and is object code compatible with the AAMP2 processor [4]. The CAPS/AAMP family of 
microprocessors has been widely used by the commercial and military aerospace industries. Some examples 
of use of earlier members of the family include: 

1. Boeing 747-400 Integrated Display System (IDS) 

2. Boeing 737-300 Electronic Flight Instrumentation System (EFIS) 

3. Boeing 777 Flight Control Backdrive 

4. Boeing 757,767 Autopilot Flight Director System (AFDS) 
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5. military and commercial Global Positioning (GPS) Systems. 

The first phase of the project consisted of the formal specification of the AAMP5 instruction set and mi- 
croarchitecture using SRI’s PVS [90, 91, 11, 69, 68, 87]. While formally specifying the microprocessor, two 
design errors were discovered in the microcode. These errors were uncovered as a result of questions raised 
by the formal methods researchers at Collins and SRI while seeking to formally specify the behavior of the 
microprocessor [59]. The Collins formal methods team believes that this effort has prevented two significant 
errors from going into the first fabrication of the microprocessor. 

The second phase of the project consisted of formally verifying the microcode of a representative subset of 
the AAMP5 instructions. Collins seeded two errors in the microcode provided to SRI in an attempt to assess 
the effectiveness of formal verification. Both of these errors (and suggested corrections) were discovered while 
proving the microcode correct [59]. It is noteworthy that both the level 2 and level 3 applications of formal 
methods were successful in finding bugs. Based on the success of the AAMP5 project, a new effort has been 
initiated with Rockwell-Collins to apply formal methods in the design level verification of a microprocessor, 
currently designated as AAMP-FV. 

3.2 Tablewise Project 

Under NASA funding, Odyssey Research Associates is working with Honeywell Air Transport Systems 
Division (Phoenix) to study the incorporation of formal methods into the company’s software development 
processes. Because Honeywell uses decision tables to specify the requirements and designs for much of their 
software 3 , ORA is developing a prototype tool, called Tablewise 4 , to analyze the characteristics of decision 
tables. Tablewise uses a generalization of Binary Decision Diagrams to determine if a particular table is 
exclusive (for every combination of parameter values, at most one action can be chosen) and exhaustive 
(for every combination of parameter values, at least one action can be chosen). The tool is also capable of 
automatically generating documentation and Ada code from a decision table [37] . We consider this a level 
3 application of formal methods: although a general purpose prover is not used, the analysis is mechanized 
in a computer program. 

In 1995, ORA will develop algorithms to handle advanced analysis of decision tables. Two particular 
areas of analysis that will be considered are testing of additional properties of tables and techniques for 
efficiently handling partitioned tables. The Honeywell personnel involved in the project hope that the 
concepts developed in the Tablewise project can be incorporated into an industrial-strength tool that will 
significantly reduce the effort required to develop new software. 

3.3 Union Switch and Signal 

As part of a joint research agreement, NASA Langley formal methods researchers are collaborating with 
engineers at Union Switch and Signal (US&S) to use formal methods in the design of railway switching 
and control applications. Railway switching control systems, like digital flight control systems, are safety 
critical systems. US&S is the leading U.S. supplier of railway switching control systems. Their Advanced 
Technology Group, lead by Dr. Joseph Profeta, has applied formal methods in past efforts and turned to 
NASA for expertise in integrating these techniques into their next generation products. 

The initial project, started in 1993, was a cooperative effort between NASA, US&S, and Odyssey Research 
Associates. The result of this first year’s work was a formal mathematical model of a railway switching 
network, defined in two levels. The top level of the model provides the mechanisms for defining the basic 
concepts: track, switches, trains and their positions and control liners of a train (i.e. how far down the 
track it has clearance to travel.) The second level is a formalization of the standard scheme used in railroad 
control, the block model control system. A level 2 proof that the fixed block control system is “safe” with 
respect to the top level model has been completed. Models of US&S proprietary control schemes were also 
formulated. 

3 A decision table is a tabular format for defining the rules that choose a particular action to perform based on the values of 
certain parameters. 

4 previously called Tbell. 



The European formal methods community has addressed safety properties of certain components of 
railroad control systems, but the work there has typically been at lower levels. The cooperative work with 
US&S is unique in that a high level model of a railroad system has been described and used to analyze the 
safety of various control schemes. 

The next phase of the collaborative effort will concentrate on formal modeling and analysis of the fault- 
tolerant core of US&S’s next generation fail-stop control architecture. 

3.4 Space Applications 

A team spread across three NASA centers has been formed to study the application and technology transfer 
of formal methods to NASA space programs. A consortium of researchers and practitioners from LaRC, JSC, 
and JPL, together with support from Loral Space Information Systems, SRI International, and ViGYAN 
Inc., has been actively pursuing this objective since late 1992. The near term goal is to define and carry 
out pilot projects using portions of existing large-scale space programs. The long term goal is to enable 
organizations such as Loral to reduce formal methods to practice on programs of national importance. 

The NASA Formal Methods Demonstration Project for Space Applications focuses on the use of formal 
methods for requirements analysis because the team believes that formal methods are more practically 
applied to requirements analysis than to late-lifecycle development phases [40]. A series of trial projects 
was conducted and cost effectiveness data were collected. The team’s efforts in 1993 were concentrated on a 
single pilot project, while later efforts in 1994 have been more diffuse. 

The 1993 project was the formal specification of a very mature piece of the Space Shuttle flight control 
requirements called Jet Select. Initial specifications were written to capture an existing, low-level statement 
of the requirements. Few proofs were produced for the first specification, but 46 issues were identified and 
several minor errors were found in the requirements. A second specification was produced for an abstract 
(i.e., high level) representation of the Jet Select requirements. This abstraction, along with the 24 proofs of 
key properties, was accomplished in under 2 work months, and although it only uncovered 6 issues, several 
of these issues were significant. 

NASA Langley’s primary role in 1994 included support for two Space Shuttle software change requests 
(CR). One CR concerns the integration of new Global Positioning System (GPS) functions while the other 
concerns a new function to control contingency aborts known as Three Engine Out (3 E/O). Both of these 
tasks involve close cooperation among formal methods researchers at NASA Langley, ViGYAN Inc., and SRI 
International with requirements analysts from Loral Space Information Systems. 

The Space Shuttle is to be retrofitted with GPS receivers in anticipation of the TACAN navigation system 
being phased out by the DoD. Additional navigation software will be incorporated to process the position 
and velocity vectors generated by these receivers. A decision was made to focus the trial formal methods 
task on just a few key areas because the CR itself is very large and complex. A set of preliminary formal 
specifications was developed for the new Shuttle navigation principal functions known as GPS Receiver State 
Processing and GPS Reference State Processing, using the language of SRI’s Prototype Verification System 
(PVS). While writing the formal specifications, 43 minor discrepancies were detected in the CR and these 
have been reported to Loral requirements analysts. 

The Three Engine Out (3 E/O) Task is executed each cycle during powered flight until either a contin- 
gency abort maneuver is required or progress along the powered flight trajectory is sufficient to preclude 
a contingency abort even if three main engines fail. The 3 E/O task consists of two parts: 3 E/O Region 
Selection and 3 E/O Guidance. 3 E/O Region Selection is responsible for selecting the type of external tank 
(ET) separation maneuver and assigning the corresponding region index. 3 E/O guidance monitors ascent 
parameters and determines if an abort maneuver is necessary. 

We have developed and analyzed a formal model of the series of sequential maneuvers that comprise the 3 
E/O algorithm. To date, 20 potential issues have been found, including undocumented assumptions, logical 
errors, and inconsistent and imprecise terminology. These findings are listed as potential issues pending 
review by the 3 E/O requirements analyst. 

The GPS and 3 E/O tasks has continued into 1995. We hope to get formal methods incorporated 
as a requirements analysis technique for Space Shuttle software. In addition, NASA Langley contributed 
to a NASA guidebook under development by the inter-center team. The first volume of the guidebook is 


intended for managers of NASA projects who will be using formal methods in requirements analysis activities. 
A second volume is planned that will be aimed at practitioners. NASA will publish the first volume early in 
1995, with the second volume expected by early 1996. 

3.5 Requirements Analysis 

Better methods for writing and analyzing requirements is one of the greatest needs that commercial indus- 
try faces today. Requirements are usually incomplete, poorly defined, and change rapidly as a system is 
developed. Errors introduced in requirements are often the most serious because they manifest themselves 
as major design errors and are often very expensive to correct when they are discovered, usually late in the 
life-cycle during implementation testing. 

NASA Langley is sponsoring work to develop special interfaces to PVS, a formal specification language 
and theorem proving environment developed by SRI. The goal of this work is to develop an interface that 
(1) is readable by Domain. Experts and typical customers, (2) is precise enough to support formal analysis, 
(3) supports concurrent development by many individuals, and (4) discourages overspecification. 

3.6 NASA Small Business Innovative Research Program 

In 1993, a formal methods subtopic was a part of the NASA Small Business Innovative Research (SBIR) 
solicitation. Two proposals were selected for 6-month Phase I funding for 1994: VHDL Lightweight Tools , by 
Odyssey Research Associates, and DRS — Derivation Reasoning System, A Digital Design Derivation System 
for Hardware Synthesis, by Derivation Systems, Inc. of Bloomington, Indiana. After the completion of the 
Phase I efforts, Derivation Systems’ proposal for Phase II funding was accepted, and contract negotiations 
are currently underway to initiate a 2- year effort. Contracts for these efforts just recently began. 

4 Past Efforts 

This section describes previous work in each of the following four focus areas: fault-tolerant systems, verifi- 
cation of software, verification of hardware devices, and civil air transport requirements specification. 

4.1 Fault-tolerant Systems 

The goal of this focus area was to create a formalized theory of fault tolerance including redundancy man- 
agement, clock synchronization, Byzantine agreement, voting, etc. Much of the theory developed here is 
applicable to future fault-tolerant systems designs. A detailed design of a fault-tolerant reliable computing 
base, the Reliable Computing Platform (RCP), has been developed and proven correct. It is hoped that 
the RCP will serve as a demonstration of the formal methods process and provide a foundation that can 
be expanded and used for future aerospace applications. It is one of the largest formal verifications ever 
performed. 

The RCP architecture was designed in accordance with a system-design philosophy called “Design For 
Validation” [42, 41]. The basic tenets of this design philosophy are as follows: 

1. A system is designed in such a manner that complete and accurate models can be constructed to 
estimate critical properties such as reliability and performance. All parameters of the model that 
cannot be deduced from the logical design must be measured. All such parameters must be measurable 
within a feasible amount of time. 

2. The design process makes tradeoffs in favor of designs that minimize the number of parameters that 
must be measured in order to reduce the validation cost. A design that has exceptional performance 
properties yet requires the measurement of hundreds of parameters (for example, by time-consuming 
fault-injection experiments) would be rejected over a less capable system that requires minimal exper- 
imentation. 



3. The system is designed and verified using rigorous mathematical techniques, usually referred to as a 
formal verification. It is assumed that the formal verification makes system failure due to design faults 
negligible so the reliability model does not include transitions representing design errors. 

4. The reliability (or performance) model is shown to be accurate with respect to the system implemen- 
tation. This is accomplished analytically not experimentally. 

Thus, a major objective of this approach is to minimize the amount of experimental testing required and 
maximize the ability to reason mathematically about correctness of the design. Although testing cannot be 
eliminated from the design/ validation process, the ‘primary basis of belief in the dependability of the system 
must come from analysis rather than from testing . 

4.1.1 The Reliable Computing Platform 

The Reliable Computing Platform dispatches control-law application tasks and executes them on redundant 
processors as illustrated in figure 1. The intended applications are safety critical with reliability requirements 



Figure 1: Intended Application of RCP 

on the order of 1 — 10~ 9 . The reliable computing platform performs the necessary fault-tolerant functions 
and provides an interface to the network of sensors and actuators. 

The RCP operating system provides the applications software developer with a reliable mechanism for 
dispatching periodic tasks on a fault-tolerant computing base that appears to him as a single ultrareliable 
processor. A multi-level hierarchical specification of the RCP is shown in figure 2. 

The top level of the hierarchy describes the operating system as a function that sequentially invokes 
application tasks. This view of the operating system will be referred to as the uniprocessor specification (US), 
which is formalized as a state transition system and forms the basis of the specification for the RCP. Fault 
tolerance is achieved by voting results computed by the replicated processors operating on the same inputs. 
Interactive consistency checks on sensor inputs and voting of actuator outputs require synchronization of the 
replicated processors. The second level in the hierarchy (RS) describes the operating system as a synchronous 
system where each replicated processor executes the same application tasks. The existence of a global time 
base, an interactive consistency mechanism and a reliable voting mechanism are assumed at this level. 

Level 3 of the hierarchy (DS) breaks a frame into four sequential phases. This allows a more explicit 
modeling of interprocessor communication and the time phasing of computation, communication, and voting. 
At the fourth level (DA), the assumptions of the synchronous model must be discharged. Rushby and von 
Henke [79] report on the formal verification of Lamport and Melliar-Smith’s [50] interactive-convergence clock 
synchronization algorithm. This algorithm can serve as a foundation for the implementation of the replicated 
system by bounding the amount of asynchrony in the system so that it can duplicate the functionality of the 
DS model. Dedicated hardware implementations of the clock synchronization function are a long-term goal. 

In the LE model, a more detailed specification of the activities on a local processor are presented. In 
particular, three areas of activity are elaborated in detail: (1) task dispatching and execution, (2) minimal 
voting, and (3) interprocessor communication via mailboxes. An intermediate model, DA_minv, that sim- 
plified the construction of the LE model was used. Some of the refinements occur in the DA_minv model 
and some in the LE model. For example, the concept of minimal voting is addressed in considerable detail 
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Figure 2: Hierarchical Specification of the Reliable Computing Platform. 


in the DA_minv model. Of primary importance in the LE specification is the use of a memory management 
unit by the local executive in order to prevent the overwriting of incorrect memory locations while recovering 
from the effects of a transient fault. 

Figure 3 depicts the generic hardware architecture assumed for implementing the replicated system. 
The hardware architecture is a classic N-modular redundant (NMR) system with a small number, N, of 
processors. Single-source sensor inputs are distributed by special purpose hardware executing a Byzantine 
agreement algorithm. Replicated actuator outputs are all delivered in parallel to the actuators, where force- 
sum voting occurs. Interprocessor communication links allow replicated processors to exchange and vote on 
the results of task computations. 

The top two levels of the RCP were originally formally specified in standard mathematical notation and 
connected via mathematical (i.e. level 2 formal methods) proof [25, 24, 22]. Under the assumption that a 
majority of processors is working in each frame, the proof establishes that the replicated system computes 
the same results as a single processor system not subject to failures. Sufficient conditions were developed 
that guarantee that the replicated system recovers from transient faults within a bounded amount of time. 
SRI subsequently generalized the models and constructed a mechanical proof in Ehdm [75]. Next, the local 
team developed the third and fourth level models. The top two levels and the two new models (i.e. DS and 
DA) were then specified in Ehdm and all of the proofs were done mechanically using the Ehdm 5.2 prover 
[12,23]. 

Both the DA_minv model and the LE model were specified formally and have been verified using 
the Ehdm verification system[13]. All RCP specifications and proofs are available electronically via the 
Internet using anonymous FTP or World Wide Web (WWW) access. Anonymous FTP access is avail- 
able through the host deduction.laxc.nasa.gov using the path pub/fm/larc/RCP-specs. WWW ac- 
cess to the FTP directory is provided through the NASA Langley Formal Methods Program home page: 
http : //atb-www . larc . nasa . gov/f m-top . html 

4.1.2 Clock Synchronization 

The redundancy management strategies of virtually all fault-tolerant systems depend on some form of voting, 
which in turn depends on synchronization. Although in many systems the clock synchronization function 
has not been decoupled from the applications (e.g. the redundant versions of the applications synchronize 
by messages), research and experience have led us to believe that solving the synchronization problem 
independently from the applications design can provide significant simplification of the system [49, 32]. The 
operating system is built on top of this clock-synchronization foundation. Of course, the correctness of 
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Figure 3: Generic hardware architecture. 


this foundation is essential. Thus, the clock synchronization algorithm and its implementation are prime 
candidates for formal methods. The verification strategy shown in figure 4 is being explored. 



Figure 4: Hierarchical Verification of Clock Synchronization 
The top-level in the hierarchy is an abstract property of the form: 

V non-faulty p,q : \C p (t) - C q (t)\ < 6 

where 6 is the maximum clock skew guaranteed by the algorithm as long as a sufficient number of clocks 
(and the processors they are attached to) are working. The function C p (t) gives the value of clock p at real 
time t. The middle level in the hierarchy is a mathematical definition of the synchronization algorithm. The 
bottom level is a detailed digital design of a circuit that implements the algorithm. The bottom level is 
sufficiently detailed to make translation into silicon straight forward. 

The verification process involves two important steps: (1) verification that the algorithm satisfies the 
maximum skew property and (2) verification that the digital circuitry correctly implements the algorithm. 
The first step was completed by SRI International. The first such proof was accomplished during the design 
and verification of SIFT [50]. The proof was done by hand in the style of journal proofs. More recently this 
proof step was mechanically verified using the Ehdm theorem prover[79, 80]. In addition, SRI mechanically 
verified Schneider’s clock synchronization paradigm [81] using Ehdm[88, 89]. A further generalization was 
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found at NASA Langley [60] 5 . The design of a digital circuit to distribute clock values in support of 
fault-tolerant synchronization was completed by SRI and was partially verified. 6 CLI reproduced the SRI 
verification of the interactive convergence algorithm using the Boyer-Moore theorem prover [100]. 

NASA Langley researchers designed and implemented a fault-tolerant clock synchronization circuit ca- 
pable of recovery from transient faults [62, 61, 60]. The top-level specification for the design is the Ehdm 
verification of Schneider’s paradigm. The circuit was implemented with programmable logic devices (PLDs) 
and FOXI fiber optic communications chips [63] . 

Using a combination of formal techniques, a verified clock synchronization circuit design has also been 
developed[64]. The principal design tool was the Digital Design Derivation system (DDD) developed by 
Indiana University [8]. Some design optimizations that were not possible within DDD were verified using 
PVS. 


4.1.3 Byzantine Agreement Algorithms 

Fault-tolerant systems, although internally redundant, must deal with single-source information from the 
external world. For example, a flight control system is built around the notion of feedback from physical 
sensors such as accelerometers, position sensors, and pressure sensors. Although these can be replicated (and 
they usually are), the replicates do not produce identical results. To use bit-by-bit majority voting, all of 
the computational replicates must operate on identical input data. Thus, the sensor values (the complete 
redundant suite) must be distributed to each processor in a manner which guarantees that all working 
processors receive exactly the same value even in the presence of some faulty processors. This is the classic 
Byzantine Generals problem [51]; algorithms to solve the problem are called Byzantine agreement algorithms. 
CLI investigated the formal verification and implementation of such algorithms. They formally verified the 
original Marshall, Shostak, and Lamport version of this algorithm using the Boyer Moore theorem prover 
[5]. They also implemented this algorithm down to the register-transfer level and demonstrated that it 
implements the mathematical algorithm [6], and then subsequently verified the design down to a hardware 
description language HDL developed at CLI [66]. A more efficient mechanical proof of the oral messages 
algorithm was also developed by SRI[76]. 

ORA also investigated the formal verification of Byzantine Generals algorithms. They focused on the 
practical implementation of a Byzantine-resilient communications mechanism between Mini-Cayuga micro- 
processors [92, 7]. The Mini-Cayuga is a small but formally verified microprocessor developed by ORA. It is 
a research prototype and has not been fabricated. The communications circuitry would serve as a foundation 
for a fault-tolerant architecture. It was designed assuming that the underlying processors were synchronized 
(say by a clock synchronization circuit). The issues involved with connecting the Byzantine communications 
circuit with a clock synchronization circuit and verifying the combination has not yet been explored. 

Thambidurai and Park [94] introduced a fault model that classified faults into three categories: asym- 
metric, symmetric, and benign. They further suggested the need for and developed an algorithm that had 
capabilities beyond that of the earlier Byzantine generals algorithms. In particular, their algorithm can mask 
the effects of a less severe class of faults, in a more effective way. SRI has formally verified an improved 
version of this algorithm [55, 54, 56] 

The newly developed hybrid-fault theory was then applied to the analysis of the Charles Stark Draper 
Labs “Fault-Tolerant Processor” (FTP). A unique feature of this architecture is its use of “interstages” 
to relay messages between processors. These are significantly smaller than a processor and lead to an 
asymmetric architecture that is far more efficient than the traditional Byzantine agreement architectures. 
The SRI work not only formalized the existing informal analysis but extended it to cover a wider range of 
faulty behavior [57]. 

Also SRI subsequently generalized their clock synchronization work to encompass the hybrid fault model 
[78]. 


4.2 Verification of Software 

Our past software verification projects are described in this section. 

5 The bounded delay assumption was shown to follow from the other assumptions of the theory. 

6 Unlike the NASA circuit, the SRI intent is that the convergence algorithm be implemented in software. 
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4.2.1 Formal Specification of Space Shuttle Jet Select 

NASA Langley worked with NASA Johnson Space Center and the Jet Propulsion Laboratory (JPL) in a study 
to explore the feasibility and utility of applying mechanically-supported formal methods to requirements 
analysis for space applications. The team worked jointly to develop a formal specification of the Jet Select 
function of the NASA Space Shuttle, which is a portion of the Shuttle’s Orbit Digital Auto-Pilot (DAP). 
Specifications were written at three different levels of abstraction. The highest level specifications were 
proved to meet a set of critical properties. This formal analysis uncovered hidden problems in a highly 
critical and mature FSSR specification for Shuttle. This project impressed several key members of the 
Shuttle software community that the benefits of formal methods are concrete and economically realizable. 
A very favorable reaction was received from the IBM (now Loral) requirements analysts and senior JSC 
personnel (Bob Hinson, in particular). They would like to work with us “to build a different paradigm where 
engineers write requirements like this before passing the requirements to software development.” 

This demonstration project was funded by the Office of Safety and Mission Quality at NASA Headquar- 
ters, which controls funding for verification and validation of all major NASA space projects. 

4.2.2 Honeywell Navigation Specification 

A cooperative research effort was initiated in 1993 with Honeywell Air Transport Systems Division (Phoenix) 
to study the incorporation of formal methods into the company’s software development processes. In the 
initial project in this effort, NASA Langley funded ORA to identify a component of the Boeing 777 sys- 
tem to which formal specification techniques could be applied, and to develop the formal specifications for 
that component. ORA, in collaboration with personnel from Langley and Honeywell, chose the navigation 
subsystem as a suitable application. 

Using documents supplied to them by Honeywell, ORA developed a specification that addressed the 
following aspects of navigation: 

• basic mathematical concepts such as functions over the reals, and physical units such as distance, 
velocity, and acceleration 

• definition of objects such as aircraft, radios, sensors, navigation aids, and the navigation database 

• definition of algorithms such as complementary filter processing, navigation aid selection, navigation 
mode selection, and position determination 

• relating the mathematical model to Ada by partitioning the system in Ada package specifications, and 
annotating individual Ada functions and procedures with formal specifications 

The specification was done using ORA’s Penelope tool. 

4.2.3 Verification of Existing Ada Applications Software 

Odyssey Research Associates completed two tasks applying their Ada verification tools to aerospace appli- 
cations. The first task was to verify some utility routines obtained from the NASA Goddard Space Flight 
Center and the NASA Lewis Research Center using their Ada Verification Tool named Penelope [33]. This 
task was accomplished in two steps: (1) formal specification of the routines and (2) formal verification of the 
routines. Both steps uncovered errors [26]. The second task was to formally specify the mode-control panel 
logic of a Boeing 737 experimental aircraft system using Larch (the specification language used by Penelope) 

[34]. 


4.3 Verification of Hardware Devices 

Our past research and technology transfer efforts in the area of formal verification of hardware devices are 
described below. 


4.3.1 Boeing Hardware Devices 

The Boeing Company was contracted by NASA Langley to develop advanced validation and verification 
techniques for fly-by-wire systems. As part of the project, Boeing explored the use of formal methods. The 
goal of this work was two-fold: (1) technology transfer of formal methods to Boeing, and (2) assessment of 
formal methods technology maturity. 

The first phase of this project focused on the formal verification of “real” hardware devices using the 
HOL hardware verification methodology. With the assistance of a subcontract with U. C. Davis, Boeing 
partially verified a set of hardware devices, including a microprocessor [98], a floating-point coprocessor 
similar to the Intel 8087 but smaller [72, 71], a direct memory access (DMA) controller similar to the Intel 
8237A but smaller[46], and a set of memory-management units[86, 83]. U. C. Davis also developed the 
generic-interpreter theory to aid in the formal specification and verification of hardware devices[99, 97, 96], 
and a horizontal-integration theory for composing verified devices into a system[85, 84, 73, 45]. After 
demonstrating the feasibility of verifying standard hardware devices, Boeing applied the methodology to a 
proprietary hardware device called the Processor Interface Unit (PIU) that is being developed for aeronautics 
and space applications[29]. 

Boeing and U.C. Davis also performed an assessment of the U.K. Royal Signals and Radar Establishment’s 
(RSRE) VIPER chip [53]. This was part of a now-completed 3 year Memorandum of Understanding (MOU) 
with RSRE. CLI and Langley researchers also performed assessments of the VIPER project[10, 19, 17]. 

Application of formal methods to the suite of Intel-like devices and the PIU demonstrated that formal 
methods can be practically applied to the digital hardware devices being developed by Boeing today and 
provided insight on how to make the process more cost effective. 

4.3.2 CSDL Scoreboard Hardware 

A joint project between ORA and Charles Stark Draper Laboratory (CSDL) was completed in 1993. NASA 
Langley and the Army had been funding CSDL to build fault-tolerant computer systems for over two decades. 
During this project, CSDL became interested in the use of formal methods to increase confidence in their 
designs. ORA was given the task of formally specifying and verifying a key circuit (called the scoreboard) of 
the Fault- Tolerant Parallel Processor (FTPP) [36] in Clio [93]. The formal verification uncovered previously 
unknown design errors. When the scoreboard chip was fabricated, it worked without any error manifestation. 
It was the first time that CSDL produced a chip that worked “perfectly” on a first fabrication. CSDL credits 
VHDL-devclopment tools and formal methods for the success. 

4.3.3 Asynchronous Communication 

CLI developed a formal model of asynchronous communication and demonstrated its utility by formally ver- 
ifying a widely used protocol for asynchronous communication called the bi-phase mark protocol, also known 
as “Bi-$-M ,” “FM” or “single density” [65]. It is one of several protocols implemented by microcontrollers 
such as the Intel 82530 and is used in the Intel 82C501AD Ethernet Serial Interface. 

4.3.4 Digital Design Derivation 

Funded in part by a NASA Langley Graduate Student Research Program fellowship, Bhaskar Bose devel- 
oped the Digital Design Derivation system (DDD) and used it to design a verified microprocessor. DDD 
implements a formal design algebra that allows a designer to transform a formal specification into a cor- 
rect implementation [8]. Bose formally derived the DDD-FM9001[9] microprocessor from Hunt’s top-level 
specification of the FM9001 microprocessor[39]. 

4.4 Civil Air Transport Requirements Specification 

Work with Boeing to develop a prototype interface for formal requirements analysis of a civil air transport 
was completed in 1992[27, 28]. This work, performed under a subcontract to California Polytechnic State 
University, included development of a Wide-Spectrum Requirements Specification Language (WSRSL) and 
prototype tools to support the language. Portions of a set of requirements for an Advanced Subsonic Civil 



Transport (ASCT) developed by a Boeing engineer under previous NASA funding were rewritten in WSRSL 
to demonstrate the use of the language and toolset. Since WSRSL is a formal language, the specifications 
can be formally analyzed for syntactic correctness, completeness, and consistency. 

5 Summary 

The NASA Langley program in formal methods has two major goals: (1) develop formal methods technology 
suitable for a wide range of aerospace designs and (2) facilitate technology transfer by initiating joint projects 
between formal methods researchers and aerospace industries to apply the results of the research to real 
systems. Starting in 1991, NASA Langley initiated several aggressive projects designed to move FM into 
productive use in the aerospace community: 

• Boeing PIU Project (1991) 

• Charles Stark Draper FTPP Scoreboard Project (1991) 

• Allied Signal Hybrid Fault Models (1992) 

• Shuttle Tile Project (1992) 

• Space Shuttle Jet Select Project (1993) 7 

• Honeywell Navigation (1993) 

• Rockwell Collins AAMP5 (1993) 

• Honeywell Tablewise (1994) 

• Union Switch and Signal (1994) 

• Rockwell Collins AAMP-FV (1995) 

NASA’s program has advanced aerospace- related formal methods in the United States to the point where 
commercial exploitation of formal methods is near. Our program has driven the development of PVS, the 
most advanced general-purpose theorem prover in the world [70], and the Odyssey Research Associates 
VHDL- verification tool. Commercial industry has been anxious to work with our team, although we have 
not had sufficient resources to work with a 3 many as we would have liked. Nevertheless, we have helped lay 
the necessary foundation for productive use of formal methods in several companies. 

Fundamental research has been performed in the design and verification of a fault-tolerant reliable com- 
puting platform that can support real-time control applications. Also much progress has been made in 
developing and demonstrating formal methods for critical subsystems of the RCP such as clock synchroniza- 
tion, Byzantine agreement, and voting. 
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